(D 


TECHNICAL  REPORT  87-016 

THE  EVOLUTION  OF  TEAMWORK  SKILLS: 

AN  EMPIRICAL  ASSESSMENT 
WITH  IMPLICATIONS  FOR  TRAINING 


NOVEMBER  1987 

Albert  8.  Gllckman 
Seth  Zimmer 
R.  Craig  Montaro 
Paula  J.  Quaratta 
Wanda  J.  CampbaH 
Ban  B.  Morgan,  Jr. 
Eduardo  Salaa 


Praparad  for: 

OFFICE  OF  NAVAL  RESEARCH 
and 

HUMAN  FACTORS  DIVISION 
Naval  Training  Systems  Cantor 


DOD  DISTRIBUTION  STATEMENT 

Approved  for  public  release; 
distribution  is  unlimited. 


W.A.  RIZZO,  Head--, 
Human  Factors  Division 


H.C.  OKRASKI,  Director 
Research  and  Development 
Department 


Technical  Report  87-016 


GOVERNMENT  RIGHTS  Hi  DATA  STATEMENT 

Reproduction  of  this  publication  in  whole  or  in 
part  is  permitted  for  any  purpose  of  the  united 
States  Government. 


la.  REPORT  SECURITY  CLASSIFICATION 
nclassif ied 


2a.  SECURITY  CLASSIFICATION  AUTHORITY 


REPORT  DOCUMENTATION  PAGE 


lb  RESTRICTIVE  MARKINGS 


2b.  DECLASSIFICATION/ DOWNGRADING  SCHEDULE 


4.  PERFORMING  ORGANIZATION  REPORT  NUMBER(S) 


3.  DISTRIBUTION /AVAILABILITY  OF  REPORT 

Approved  for  public  release; 
distribution  unlimited. 


S.  MONITORING  ORGANIZATION  REPORT  NUMBER(S) 


Technical  Report,  TR87-016 


6a.  NAME  or  PERFORMING  ORGANIZATION  1 6b.  OFFICE  SYMBOL  I  7a.  NAME  OF  MONITORING  ORGANIZATION 

Center  for  Aoolied  I  applicable)  i 


Center  for  Applied 
Psychological  Studies 


6c.  ADDRESS  (Gty.  State,  and  ZIP  Code) 

Old  Dominion  University 
Norfolk,  VA  23529-0267 


Office  of  Naval  Research 


7b.  ADDRESS  (Cfty,  State,  and  ZIP  Code) 

800  North  Quincy  Street 
Arlington,  VA  22217-5000 


Bb.  OFFICE  SYMBOL 
(It  applkable) 

Code  712 


9.  PROCUREMENT  INSTRUMENT  IDENTIFICATION  NUMBER 


10.  SOURCE  OF  FUNDING  NUMBERS 


PROGRAM 

PROJECT 

TASK 

ELEMENT  NO. 

NO. 

NO 

62233N 

W3331 

5716 

14.  DATE  OF  REPORT  (Year,  Month,  Day) 

871115 


15.  PAGE  COUNT 


8a.  NAME  OF  FUNDING  /SPONSORING 

organization  Naval  Training 

Systems  Center 


8c.  ADDRESS  (City,  State,  and  ZIP  Cod*) 

Human  Factors  Division 
Department  of  the  Navy 
Orlando,  FL  32826 


11.  TITLE  (Include  Security  Clauifkatlon) 

The  Evolution  of  Teamwork  Skills:  An  Empirical  Assessment  with 
Implications  for  Trainin 


12.  personal  author(S)  Albert  s.  Glicjcman,  setn  Zimmer,  R.  Craig  Montero, 

Paula  J.  Guerette,  Wanda  J.  Campbell,  Ben  B.  Morgan,  Jr.,  Eduardo  Salas 


13a.  TYPE  OF  REPORT  Il3b  TIMF.  COVEREO 

Final  1  from  9/86  to  9/87 


16.  supplementary  notation  mis  report  was  prepared  under  tne  Navy  Manpower, 

Personnel  and  Training  R&D  Program  of  the  Chief  of  Naval  Research  under 
Contract  N000014-86-K-0732 . 


17.  _  COS  ATI  CODES _ I  18.  SUBJECT  TERMS  (Continue  on  reverie  if  neceuary  and  Identify  by  bloc*  number) 

field  I  croup  I  subgroup  I  Team  Training,  Team  Performance,  Training 

Intervention,  Training  Analysis,  Team  Develop¬ 
ment,  Teamwork 


1.SL4BSTRACT  (Continue  on  reverie  If  necessary  and  identify  by  block  number) 

7^  The  findings  reported  here  represent  the  results  of  the  first  phase 
of  a  three-year  program  of  research  designed  to  examine  factors  that 
influence  the  development  of  teamwork  during  training.  The  current  study 
was  conducted  at  the  Naval  Gunfire  Support  ( NGFS )  Department,  Little  Creek 
Amphibious  Base,  Norfolk,  Virginia.  It  represents  one  component  of  a 
research  effort  seeking  to  document  the  processes  involved  in  Team 
Evolution  and  Maturation  (TEAM). 

The  primary  objectives  of  this  research  are  to:  (1)  delineate  the 
process  variables  that  comprise  "teamwork";  (2)  develop  a  set  of  instru¬ 
ments  to  measure  these  variables;  (3)  study  how  teamwork  develops  (i.e. 
liow  teams  evolve  and  mature)  over  the  course  of  training;  and  (4)  develop 
means  to  diagnose,  correct,  and  enhance  team  performance  in  training. 

(CONTINUED)  —  -  ^ 


20  DISTRIBUTION /AVAILABILITY  OF  ABSTRACT 
□  UNCLASSIFIED/UNLIMITED  GO  SAME  AS  RPT  Q  OTIC  USERS 


22«  NAME  OF  RESPONSIBIE  INDIVIDUAL  |22t  TCI '‘PHONE  (Include  Area  Code)  22c  OFFICE  SYMBOL 

Eduardo  Salas  and  James  Lester  I  (407)  380-4651 


OD  FORM  1473, 84  MAR  83  APR  edition  may  be  u»#d  until  exhausted.  SECURITY  CLASSIFICATION  OF  THIS  PAGE 

.<*»*.  UNCLASSISiWU-  — i o~.  «««. 


21.  ABSTRACT  SECURITY  CLASSIFICATION 

Unclassified 


BJ,ooJt-i9-'tcontinued ) 

A  model  of  team  evolution  and  maturation  was  developed  based  on 
existing  models  and  methodologies  in  the  team  performance/team  training 
literature.  This  model  hypothesizes  that  a  team  progresses  through  several 
different  stages  of  development  during  the  course  of  training,  and  that  the 
speed  and  sequence  of  these  stages  is  affected  by  the  efficacy  of  the 
training  program.  In  addition,  the  model  postulates  two  separate  develop- 
-^mental  tracks — a  "taskwork"  track  and  a  "teamwork"  track.  In  order  for 
training  to  be  successful  and  for  the  group  of  individuals  to  function  as  a 
cohesive  team,  these  two  tracks  must  be  separately  developed  and  ultimately 
converged .  ( £tXo)  <z— - 

Initial  activities  at  NGFS  involved  the  development  and  refinement  of 
instruments  designed  to  measure  changes  in  teamwork  behaviors  that  occur 
during  the  course  of  training.  The  results  of  these  efforts  were  a  series 
of  measurement  devices  including  a  Critical  Team  Behaviors  Form,  a  Trainee 
Questionnaire,  and  Demographics  Form,  a  Gunnery  Liaison  Officer's  (GLO's) 
Individual  Performance  Form,  and  an  Instructor's  Individual  and  ovevall 
Performance  Forms. 

Data  were  collected  on  a  total  of  13  Navy  teams.  These  were  cate¬ 
gorized  as  more  and  less  effective  teams  according  to  an  independent 
criterion  (i.e.,  final  exam  score  during  training)  and  analyzed.  Findings 
support  the  TEAM  model  and  validate  the  sensitivity  of  the  measurement 
instruments  to  detect  both  discrimination  effects  and  evolution  and  matur¬ 
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research  in  other  team  training  settings  is  needed  to  determine  the  general- 
izability  of  the  present  findings. 
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EXECUTIVE  SUMMARY 

The  findings  reported  here  represent  the  results  of  the 
first  phase  of  a  three-year  program  of  research  designed  to 
examine  factors  that  influence  the  development  of  teamwork  during 
training.  The  current  study  was  conducted  at  the  Naval  Gunfire 
Support  (NGFS)  Department,  Little  Creek  Amphibious  Base,  Norfolk, 
Virginia.  It  represents  one  component  of  a  research  effort 
seeking  to  document  the  processes  involved  in  Team  Evolution  And 
Maturation  (TEAM) . 

The  primary  objectives  of  this  research  are  to:  (1) 
delineate  the  process  variables  that  comprise  ’•teamwork" ;  (2) 
develop  a  set  of  instruments  to  measure  these  variables;  (3) 
study  how  teamwork  develops  (i.e.,  how  teams  evolve  and  mature) 
over  the  course  of  training;  and  (4)  develop  ways  to  diagnose, 
correct,  and  enhance  team  performance  in  training. 

A  model  of  team  evolution  and  maturation  was  developed  based 
on  existing  models  and  methodologies  in  the  team  performance/ team 
training  literature.  This  model  hypothesizes  that  a  team 
progresses  through  several  different  stages  of  development  during 
the  course  of  training,  and  that  the  speed  and  sequence  of  these 
stages  is  affected  by  the  efficacy  of  the  training  program.  in 
addition,  the  model  postulates  two  separate  developmental  tracks 
— a  "taskwork"  track  and  a  "teamwork"  track.  In  order  for 
training  to  be  successful  and  for  the  group  of  individuals  to 
function  as  a  cohesive  team,  these  two  tracks  must  be  separately 
developed  and  ultimately  converged. 

Initial  activities  at  NGFS  involved  the  development  and 
refinement  of  instruments  designed  to  measure  changes  in  teamwork 
behaviors  that  occur  during  the  course  of  training.  The  results 
of  these  efforts  were  a  series  of  measurement  devices,  including 
a  Critical  Team  Behaviors  Form,  a  Trainee  Questionnaire,  a 
Demographics  Form,  a  Gunnery  Liaison  Officer's  (GLO's)  Individual 
Performance  Form,  and  an  Instructor’s  Individual  and  Overall 
Performance  Forms. 

Data  were  collected  on  a  total  of  13  Navy  teams.  These  were 
categorized  as  more  and  less  effective  teams  according  to  an 
independent  criterion  (i.e.,  final  exam  scores  during  training) 
and  analyzed.  Findings  support  the  TEAM  model  and  validate  the 
sensitivity  of  the  measurement  instruments  to  detect  both 
discrimination  effects  and  evolution  and  maturation  effects.  In 
addition,  the  findings  suggest  the  presence  of  a  "taskwork" 
factor,  a  "teamwork"  factor,  and  a  "jelling"  factor,  which  occur 
as  separate  factors  at  the  start  of  training  but  which  converge 
during  the  final  stage  of  training.  Based  on  these  results, 
preliminary  interventions  designed  to  improve  team  training 
effectiveness  are  suggested.  Further  research  in  other  team 
training  settings  is  needed  to  determine  the  general izability  of 
the  present  findings. 
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THE  EVOLUTION  OF  TEAMWORK  SKILLS: 

AN  EMPIRICAL  ASSESSMENT  WITH  IMPLICATIONS  FOR  TRAINING 

"Of  all  the  missions  a  ship  is  required  to  accomplish, 
none  require  greater  teamwork  than  NGFS.  To  become  proficient 
at  this  demanding  task,  both  formal  training  and  frequent 
practice  are  required"  ( COMNAVSURFLANTINST  *5570. 2C,  1982). 


OVERVIEW 

This  report  describes  a  comprehensive  investigation  of  the 
factors  that  contribute  to  the  development  of  teamwork  during  the 
operational  training  of  Navy  teams.  The  investigation  was 
conducted  at  the  Naval  Gunfire  Support  (NGFS)  Department, 
Little  Creek  Amphibious  Base,  Norfolk,  Virginia.  It  represents 
one  component  of  a  broader  program  of  research  that  has  been 
designed  to  investigate  learn  Evolution  &nd  Maturation  (TEAM) . 
The  primary  goals  of  this  research  program  are  to: 

1)  delineate  the  process  variables  that  comprise  "teamwork"; 

2)  develop  a  set  of  instruments  to  measure  these  variables; 

3 )  increase  the  understanding  of  how  teamwork  develops  ( i . e . , 
how  teams  evolve  and  mature)  over  the  course  of 
training;  and 

4)  develop  ways  to  diagnose,  correct,  and  enhance  team 
performance  in  training  (thus  enhancing  the  efficiency 
and  effectiveness  of  team  training  systems) . 

Earlier  work  in  this  program  has  provided  a  thorough  review 
of  the  relevant  literature,  the  articulation  of  a  model  of  team 
development,  instruments  to  measure  team  evolution  and 
maturation,  and  preliminary  observations  of  teams  undergoing 
training  at  NGFS  (all  related  to  goals  1  and  2  above) . 
Descriptions  and  results  of  these  prior  efforts  have  been 
reported  by  Morgan,  Glickman,  Woodard,  Blaiwes,  &  Salas  (1986). 


ORGANIZATION  OF  THE  REPORT 

The  present  report  presents  the  results  obtained  with 
data  collected  at  NGFS  and  discusses  the  implications  of  the 
findings  for  improving  team  training  systems  (related  to  goals 
3  and  4  above) .  These  results  provide  an  initial  appraisal  of 
the  validity  of  the  instruments  and  procedures  developed  in  the 
earlier  phases  of  the  research.  In  addition,  they  document  the 
nature  of  the  changes  in  team  behaviors  and  their  impact  on  the 
effectiveness  of  team  performance.  They  also  provide  a  basis  for 
suggestions  of  ways  in  which  NGFS  training  can  be  improved.  When 
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combined  with  the  results  of  later  phases  of  this  research 
program,  the  current  data  will  provide  a  basis  for  the 
construction  of  new  training  aids,  programs,  or  instructional 
approaches  of  more  general  applicability  that  will  add  to  the 
quality  of  team  training  in  the  Navy. 


The  introductory  section  of  the  report  discusses  the  factors 
that  have  contributed  to  the  development  and  design  of  the 
present  research  effort.  This  includes  a  brief  discussion  of  the 
knowledge  gaps  that  have  resulted  from  prior  research,  the  types 
of  questions  which  stimulated  the  present  research,  and  a  summary 
recapitulation  of  the  theoretical  and  descriptive  models  upon 
which  the  current  investigation  is  based.  The  method  section 
describes  the  various  measurement  instruments  and  procedures  used 
to  collect  data  at  NGFS .  The  results  section  presents  an 
overview  of  the  data  collected  and  describes  evidence  which 
indicates  the  presence  of  evolution  and  maturation  effects  in  the 
NGFS  teams.  This  section  also  illustrates  the  fact  that  the 
measurements  provide  a  basis  for  discriminating  between  more 
effective  and  less  effective  teams.  It  ends  with  a  summary  which 
highlights  the  most  important  findings.  The  discussion  section 
provides  additional  insights  concerning  the  implications  of  the 
current  findings  as  they  relate  to  the  theoretical  and  conceptual 
framework,  the  measurement  techniques,  and  the  analyses  employed 
in  the  current  investigation.  In  addition,  the  results  are 
discussed  in  terms  of  their  application  to  team  training 
systems  design  and  development  in  the  Navy.  Finally,  specific 
training  interventions  are  suggested  for  further  investigation 
and  evaluation  during  the  next  stage  of  the  research  program. 


INTRODUCTION 

This  research  focuses  on  the  Investigation  of 
operational  Navy  teams  undergoing  prescribed  training  in 
state-of-the-art  simulation  facilities.  It  is  based  on  the 
supposition  that  a  fuller  understanding  of  the  behaviors  that 
represent  the  evolution  and  maturation  of  teamwork  in  such  teams 
will  contribute  substantially  to  the  Navy's  overall  effort  to 
maximize  the  effectiveness  of  its  team  training  systems. 

For  purposes  of  this  research,  "team"  has  been  defined  as 
"a  distinguishable  set  of  two  or  more  individuals  who  interact 
interdependently  and  adaptively  to  achieve  specified,  shared,  and 
valued  objectives"  (Morgan,  et  al.,  1986).  Similarly,  teamwork 
is  defined  as  consisting  of  those  actions,  processes,  and 
behaviors  which  contribute  to  a  team's  ability  to  interact 
interdependently  and  adaptively  in  order  to  achieve  specific, 
shared,  and  valued  objectives  in  a  most  efficient  manner. 
This  includes  behaviors  related  to  the  development  of  effective 
team  communication,  coordination,  cohesion,  cooperation,  etc. 
Teamwork  is  considered  to  be  enhanced  through  purposefully 
committed  person-to-person  activities  designed  to  improve 
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operational  task  performance,  communication  skills,  and 
interaction  patterns  within  the  team. 


BACKGROUND 

Earlier  research  (e.g.,  Goodstadt,  Frey  &  Glickman,  1975) 
has  shown  that  the  relationships  among  social/ occupational 
variables  are  different  at  different  phases  of  the  socialization 
process  in  military  service.  Similarly,  Terborg,  Castore,  & 
Dinnino  (1976)  found  that  the  influence  of  attitude  similarity  on 
group  cohesion  was  not  immediately  apparent,  but  that  these 
effects  developed  over  time.  These  findings  support  the 
supposition  that  factors  which  influence  group  performance  must 
be  studied  over  time,  as  the  group  changes.  Nevertheless,  one  of 
the  major  shortcomings  of  previous  team  training  research  has 
been  the  failure  to  give  specific  attention  to  the  learning  of 
teamwork  skills  (cf.  Cissna,  1984;  Denson,  1981;  Dyer,  1984), 
and  to  improving  the  understanding  of  how  various  patterns  of 
team-member  interactions  develop,  change,  and  impact  performance 
during  the  life-cycle  of  the  team. 

Although  the  need  for  investigations  of  group  and  team 
development  has  been  expressed  frequently,  it  has  likewise  been 
noted  that  substantial  gaps  exist  in  the  scientific  understanding 
of  how  to  analyze,  define,  measure,  design,  and  evaluate  team 
training  and  its  components  (cf.,  Alluisi,  1977;  Baum,  Modrick,  & 
Hollingsworth,  1981;  Denson,  1981;  Goldin  &  Thorndyke,  1980;  Hall 
&  Rizzo,  1975;  Nieva,  Fleishman,  &  Reich,  1978;  Dyer,  1984).  As 
pointed  out  by  Salas,  Blaiwes,  Reynolds,  Glickman,  and  Morgan 
(1985) ,  problems  particularly  relevant  to  the  current  research 
include  the  inability  to  adequately  define  the  variables  that 
contribute  to  effective  teamwork,  a  lack  of  integrative 
conceptualizations,  inadequate  measurement  systems,  and  limited 
knowledge  of  how  teams  develop  over  time. 

One  reason  for  the  limited  understanding  of  team  development 
is  that  most  of  the  previous  research  has  dealt  with  mature 
teams.  In  addition,  inadequate  understanding  of  team  learning 
processes — particularly  as  they  relate  to  the  ways  in  which 
teams  learn  to  work  together  over  time — have  inhibited  advances 
in  the  development  of  team  training  instructional  systems. 
Research  in  this  area  has  also  been  hindered  by  the  inherent 
complexities  of  group  interactions,  the  assorted  difficulties 
involved  in  observing  teams  at  work,  and  the  practical 
constraints  imposed  upon  efforts  to  conduct  longitudinal  studies. 

In  the  Navy,  as  in  most  other  organizations,  team 
performance  has  often  been  based  on  faith  in  the  "natural 
evolution"  of  teamwork,  with  "trial  ar.d  error"  characterizing  the 
formulation  of  training  procedures.  The  example  of  athletic 
(e.g.,  basketball)  teams  is  often  cited  to  point  out  that  it 
takes  time — experience  working  with  one  another — for  individuals 
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to  learn  to  work  as  a  team,  and  that  the  coach  (i.e.,  instructor) 
makes  an  important  difference  in  the  ultimate  success  of  the 
team.  Everyone  "knows"  this,  but  there  is  relatively  little 

understanding  of  what  team  members  or  instructors  actually  do 

that  accounts  for  the  differences  between  good  and  poor  teams. 
For  the  most  part,  instructors  provide  "team"  training  only  by 
providing  team  members  with  the  opportunity  to  practice  their 
individual  skills  in  a  group  setting  (Salas  et  al.,  1985). 
Feedback  concerning  the  behaviors  that  comprise  teamwork  is 

usually  lacking.  Trainees  receive  instruction  in  teamwork,  as 

such,  only  to  the  extent  that  instructors  are  able  to  devise  ad 
hoc  waye  to  teach  them  (Morgan  et  al.,  1986). 

While  it  is  common  to  assert  that  "a  team  is  more  than  the 
sum  of  its  parts,"  relatively  little  is  known  about  the  social 
and  organizational  processes  that  make  this  so  or  about  how  roles 
and  patterns  of  interaction  emerge  and  evolve  during  training  (or 
during  the  total  life  history  of  a  team) .  Thus,  a  number  of 
questions  arise  concerning  the  nature  of  team  evolution  and 
maturation.  For  example, 

*  Which  variables  determine  how  well  a  team  does  its  work,  and 
at  what  stages  of  team  evolution  and  maturation  are  these 
variables  most  salient? 

*  What  kinds  of  teamwork  behaviors  occur  at  the  various  stages 
of  training? 

*  What  is  the  nature  of  teamwork?  How  should  it  be  defined? 
How  can  it  be  measured? 

*  what  training  variables  affect  the  rate  of  team  maturation? 

*  What  do  people  need  to  learn  about  working  with  one  another 
in  order  to  enhance  the  outcome  of  the  collective  effort? 

*  What  kind  of  teamwork  training  should  be  provided  along  with 
the  technical  skills  training  typically  provided  in  Navy 
training? 

*  What  instructional  technology  or  procedures  may  enhance 
teamwork? 

These  are  the  kinds  of  questions  that  have  provided  the 
impetus  for  the  current  research  (see  also  cissna,  1984;  Denson, 
1981;  Dyer,  1984).  Stimulated  by  questions  such  as  these, 
efforts  during  the  first  year  of  this  research  resulted  in  the 
development  of  several  models  to  guide  the  planning  and  conduct 
of  the  research.  The  first  of  these  was  an  overall  descriptive 
model  of  the  ongoing  sequence  of  processes  by  which  teams  evolve, 
mature,  and  improve  their  performance  over  time.  Two  additional 
models  were  developed  to  represent  the  specific  stages  and 
processes  involved  in  NGFS  training.  These  models  are  discussed 
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below  in  order  to  provide  a  better  underetanding  of  the 
background,  conceptual  framework,  and  objectives  for  this 
investigation.  Additional  details  concerning  the  models  and 
their  development  are  available  in  Morgan  et  al.  (1986). 


RESEARCH  OBJECTIVES 

During  the  initial  year  of  this  research  program,  a  review 
of  the  literature  was  conducted  in  order  to  identify  previous 
findings,  hypotheses,  and  concepts  concerning  the  temporal 
evolution  of  teams.  This  effort  resulted  in  the  development  of  a 
model  of  team  evolution  and  maturation  (the  TEAM  model) ,  which 
was  designed  to  represent  the  available  conceptualizations  of  how 
teams  develop  over  time,  and  to  stimulate  hypotheses  to  guide  the 
current  measurement  'of  team  evolution  and  maturation.  The 
objectives  of  the  current  investigation  can  be  illuminated  by  a 
brief  discussion  of  this  model. 

The  TEAM  model  has  its  origins  in  an  open  systems  framework 
of  organizational  effectiveness  developed  by  Glickman,  Goodstadt, 
Kavanagh  &  Bass  (1977)  .  As  illustrated  in  Figure  1,  this 
framework  considers  team  performance  as  a  dynamic  activity  of 
process-outcome  linkages  which  occur  within  an  environmental 
envelope  of  organizational  and  other  exogenous  forces. 


The  TEAM  Msdsl 

The  TEAM  model  is  presented  in  Figure  2  (for  a  detailed 
exposition  of  the  model,  see  Morgan  et  al.,  1986).  The  concepts 
embodied  in  this  model,  and  the  literature  on  which  it  is  based, 
suggest  that  :  1)  the  processes  involved  in  the  development  of  a 
team  change  over  time;  2)  these  processes  form  continuous 
process-outcome  linkages  with  intermediate  outcomes  serving  as 
inputs  for  future  processes;  3)  team  members  learn  teamwork 
skills  as  a  result  of  experience  within  the  team;  and  4)  the 
experience  and  learning  acquired  by  team  members  will  induce 
changing  behavior  patterns  over  time.  The  TEAM  model  also 
depicts  the  variables  involved  in  the  processes  by  which  members 
learn  the  demanding,  adaptive,  interdependent  actions  that  are 
required  to  achieve  the  team's  performance  objectives. 

The  TEAM  model  also  incorporates  nine  stages  of  team 
development  (i.o.,  preforming,  forming,  storming,  norming,  per- 
forming-I,  reforming,  performing-II,  conforming,  and  de-forming) 
that  have  been  suggested  by  several  other  authors  (e.g.,  Bales  & 
Strodtbeck,  1951;  Bell,  1982;  Bennis  &  Shepard,  1956;  Caple, 
1978;  Fisher,  1970;  Gersick,  1985;  Tuckman,  1965;  Tuckman  & 
Jensen,  1977)  .  Thus,  it  describes  a  series  of  developmental 
stages  through  which  task-oriented  teams  are  hypothesized  to 
evolve,  beginning  with  initial  levels  of  incompetence  and 
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Figure  1.  Dynamic  organizational  effectiveness  model. 
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This  test  site  was  selected  because  it  met  the  requirements 
of  a  set  of  previously  established  guidelines  (see  Appendix  A) 
which  were  based  on  visits  and  interviews  with  key  personnel  at 
several  prospective  research  sites.  In  addition,  several  factors 
made  NGFS  particularly  attractive  as  a  test  bed  for  the 
hypotheses  and  objectives  of  this  investigation:  (1)  Shipboard 
gunfire  support  teams  on  combat  vessels  are  required  to  begin 
their  training  cycle  with  NGFS;  (2)  The  membership  of  the  team 
is  required  to  remain  intact  during  attendance  at  the  school  and 
live  firing  exercises  scheduled  to  take  place  within  90  days 
after  school  training;  (3)  On-line  performance  scores  are 
available  from  midterm  and  final  tests  conducted  on  the  shore 
bombardment  simulator  at  the  school  and  from  subsequent 
qualification  tests  conducted  at  a  live  firing  range;  (4)  The 
teams  are  required  to  complete  training  at  the  school,  and  then 
at  the  live  firing  range,  in  order  to  be  certified  as  being  in  a 
state  of  readiness  to  deploy  with  the  fleet.  This  last  fact  is 
perhaps  most  important  from  a  motivational  standpoint.  The 
serious  repercussions  associated  with  training  failures  (e..g., 
notification  to  Congress  of  the  failure  to  attain  operational 
readiness)  provide  a  strong  incentive  for  the  ships'  commanders 
to  ensure  that  the  training  and  live-fire  requirements  are  taken 
seriously.  In  turn,  this  serves  to  increase  the  perceived 
"reality"  of  the  NGFS  training  for  the  individual  members  who  are 
assigned  for  such  training. 

At  the  time  of  this  investigation,  training  at  the  NGFS 
school  consisted  of  a  morning  orientation  (pre-exercise)  session 
(usually  on  Monday)  and  3  1/2  to  4  1/2  days  of  subsequent 
simulator  exercises  (Some  changes  have  been  made  in  the  content 
and  structure  of  NGFS  training  since  this  study  was  conducted) . 
The  simulator  training  is  divided  into  five  phases  of 
increasingly  difficult  exercises:  basics,  pre-midterm,  midterm, 
post-midterm,  and  final.  In  spite  of  the  requirement  that 
individual  team  members  come  to  training  with  a  relatively  high 
degree  of  proficiency  on  assigned  tasks,  it  was  found  that 
teams  typically  include  several  new  members  who  have  little  or 
no  actual  gunfire  support  experience.  However,  the  Navy  requires 
that  the  individuals  who  form  the  team  that  is  trained  at  NGFS 
also  constitute  the  team  that  conducts  the  live  firing  on  the 
range.  While  exceptions  may  be  made  in  the  case  of  serious 
problems  that  necessitate  the  substitution  of  team  members,  these 
must  be  made  during  the  course  of  training  at  the  school.  The 
team  that  the  school  determines  to  be  qualified  is  the  team 
that  is  expected  to  shoot  live  ammunition. 

Crews  engaged  in  NGFS  training  are  composed  of  three  teams: 
the  Bridge  team,  consisting  of  approximately  three  members;  the 
Combat  Information  Center  (CIC)  team,  consisting  of  eight 
members;  and  the  Plot  team,  consisting  of  five  members.  The  teams 
may  or  may  not  be  physically  separated  (depending  upon  the  class 
of  ship) .  Communication  between  the  teams  is  accomplished  through 
the  use  of  sound-powered  telephones.  Transmissions,  which  in 
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actual  combat  would  take  place  between  the  ship  and  the  shore- 
based  spotter  who  is  directing  the  shore  bombardment,  are 
simulated  by  the  school  staff. 

The  CIC  team  was  chosen  as  the  object  of  observation 
for  this  investigation  because  CIC  is  the  hub  of  NGFS 
activity#  and  it  requires  the  greatest  amount  of  communication, 
interdependence,  and  interaction.  It  interfaces  with  the  two 
other  shipboard  teams  as  well  as  with  the  spotter.  Furthermore, 
findings  from  the  CIC  at  NGFS  should  generalize  to  the  many  other 
Navy  teams  whose  modus  operand!  resemble  the  team  situation 
simulated  at  NGFS.  The  following  section  discusses  the  training 
of  the  CIC  team  in  terms  of  the  nature  of  the  interactions  among 
the  instructor,  the  team,  and  individual  team  members. 


Component  Model  of  NGFS  Training 

As  described  above,  NGFS  training  occurs  in  six  sequential 
phases  :  pre-exercise,  basic  missions,  pre-midterm,  midterm, 
post-midterm,  and  final.  In  addition,  the  NGFS  training  system 
consists  of  three  separate  but  interactive  components:  the 
instructor,  the  individual,  and  the  team.  In  order  to  illustrate 
the  relative  importance  of  each  component,  as  well  as  changes  in 
each  component's  importance  and  level  of  interaction  throughout 
the  course  of  training,  a  model  was  developed  to  represent  the 
activities,  contributions  and  interactions  of  each  ccr.ponent 
during  the  phases  of  NGFS  training.  This  model  is  reproduced  in 
Figure  3.  For  a  detailed  description  of  this  model  and 
subsequent  refinement  of  the  instructional  process  model,  see 
Morgan  et  al.  (1986)  and  Guerette,  Miller,  Glickman,  Morgan  and 
Salas  (1987). 

In  this  model,  the  relative  importance  of  the  instructor, 
the  individual,  and  the  team  to  the  overall  training  effort  is 
represented  by  the  size  of  the  box  containing  each  component — the 
larger  the  box,  the  more  important  the  component  during  that 
given  phase  of  training.  The  general  direction  of  flow  of 
information,  feedback,  and  other  contributions,  as  well  as  the 
relative  amount  of  each  of  these  processes,  is  indicated  by  the 
direction  and  number  of  arrows  interconnecting  the  three 
components.  Typical  behaviors  are  listed  by  phase  for  each 
component.  Instructor  behaviors  are  separated  into  those  that 
are  directed  at  the  individual  and  those  targeted  at  the  team. 

As  depicted  in  Figure  3,  it  was  hypothesized  that  at  the 
start  of  training  the  importance  of  the  individual  and  the 
instructor  is  large  relative  to  the  importance  of  the  team.  At 
this  point,  the  general  flow  of  contribution  is  from  the 
instructor  to  the  individual,  and  to  a  lesser  degree,  to  the  team 
as  a  whole.  As  training  progresses,  the  instructor's 
contribution  to  the  development  of  the  team  as  such  increases  and 
the  relative  importance  of  the  team  component  also  increases. 


Technical  Report  87-016 


25 


Technical  Report  87-016 


Gradually,  the  role  of  the  instructor  diminishes  as  he  transfers 
"ownership”  to  the  team.  Finally,  the  team  becomes  the 
dominant  component,  from  which  contributions  flow  to  both  the 
individual  and  the  instructor. 


stage.  Model  of  kgfs  Training 

As  suggested  previously  (see  Morgan  et  al.,  1986),  NGFS 
training  can  also  be  understood  in  terms  of  its  relation  to  the 
stages  of  team  development  represented  in  the  TEAM  model  (Figure 
2).  In  Figure  4,  the  six  phases  of  NGFS  training  are  presented 
in  the  format  of  the  TEAM  model,  with  the  phases  of  NGFS 
training  incorporated  beneath  the  stages  of  evolution  and 
maturation.  Vertical  lines  have  ueen  included  to  allow 
comparison  between  the  stages  of  the  TEAM  development  process 
and  the  phases  of  NGFS  training.  The  individual  circles  have 
been  labeled  with  representative  behaviors  from  each  phase.  The 
purpose  of  this  model  is  to  suggest  that  NGFS  training  will 
result  in  temporally  changing  patterns  of  team  behaviors  that  can 
be  measured  and  related  to  success  in  simulator  training. 


Summary  of  Objectives 

The  purpose  of  this  investigation  was  to  measure  the  changes 
in  team  behavior  that  contribute  most  to  the  development  of 
teamwork  skills  during  operational  training.  It  was  hypothesized 
that  changing  behavior  patterns  could  be  measured  during  NGFS 
training  and  that  these  patterns  could  be  related  to  increases  in 
teamwork  skills.  It  was  also  expected  that  these  changing 
behavior  patterns  would  reflect  changes  in  both  taskwork  and 
teamwork  activ;ties  and  that  a  convergence  of  these  skills  could 
be  demonstrated  in  successful  NGFS  teams. 

Thus,  the  current  investigation  was  designed  to  provide 
specific  information  concerning  the  evolution  and  maturation  of 
teams  in  traini.  g,  suggestions  of  ways  to  improve  team  training 
systems  in  the  Navy,  and  validation  of  the  TEAM  model  and 
measurement  procedures.  In  summary,  the  objectives  of  this 
research  were  to  (1)  identify  and  define  the  processes  and 
behaviors  that  constitute  teamwork,  (2)  demonstrate  the  utility 
of  a  set  of  instruments  designed  to  measure  thene  processes  and 
behaviors,  (3)  measure  the  evolution  and  maturation  of  NGFS  teams 
during  training,  and  (4)  develop  suggestions  of  ways  to  diagnose, 
correct,  and  enhance  team  training. 
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METHOD 

The  TEAM  measurement  procedures  were  employed  in  the 
collection  of  data  from  a  total  of  17  teams  undergoing  gunfire 
support  training  at  NGPS.  For  a  variety  of  reasons,  final  data 
sets  were  incomplete  for  four  of  these  teams  (one  team  failed  the 
midterm  examination  and  retu-.ied  to  the  ship  without  completing 
the  remaining  training  sessions,  and  schedule  conflicts 
prohibited  the  collection  of  data  for  one  or  more  sessions  with 
the  other  three  teams) .  Thus,  complete  data  sets  were  collected 
for  a  total  of  117  individuals  representing  13  teams  trained  by 
six  different  NGFS  instructors.  The  specific  data  collection 
instruments  and  measurement  procedures  employed  in  this 
investigation  are  discussed  below. 


DEVELOPMENT  OF  TEAM  MEASUREMENT  TECHNIQUES 

Based  on  direct  observations  of  team  training  and  interviews 
with  NGFS  instructors,  six  data  collection  instruments  were 
developed  for  use  in  this  investigation  (see  Morgan  et  al., 
1986) .  Three  of  these  forms  were  completed  by  the  instructors. 
These  are:  (1)  the  Critical  Team  Behaviors  Form,  which  was 
designed  to  identify  critical  behaviors  displayed  by  the 
instructor  and  the  team  members;  (2)  an  Overall  Team  Performance 
Form  which  requires  the  instructor  to  evaluate  the  overall 
performance  of  the  team  as  a  whole;  and  (3)  an  Individual 
Performance  Summary  designed  to  evaluate  the  effectiveness 
of  individual  team  members  in  the  performance  of  their 
assigned  duties. 

In  addition,  the  formal  team  leader — the  Gunnery  Liaison 
Officer  (GLO) — was  asked  to  complete  the  Individual  Performance 
Summary  in  order  to  provide  his  estimates  of  the  performances  of 
the  individual  members  of  his  team.  Two  data  collection 
instruments  were  also  completed  by  the  team  members  themselves: 
(1)  a  Self-Report  Questionnaire  which  was  used  to  record  the 
changing  perceptions  of  the  team  members  and  (2)  a  Team 
Demographics  Form  which  provided  information  on  prior  work  and 
training  experiences  related  to  NGFS  and  the  Navy  in  general.  A 
more  complete  description  of  each  of  the  measurement  instruments 
is  provided  below;  copies  of  the  forms  are  provided  in 
Appendices  B  through  G. 

With  the  exception  of  the  Demographics  Form,  which  was 
completed  on  the  afternoon  of  the  first  day  of  training,  all  of 
the  forms  were  designed  to  be  completed  after  each  morning  and 
afternoon  session  during  the  week  of  training.  In  order  to 
maximize  completeness  and  accuracy  of  recall,  all  individuals 
were  asked  to  base  their  responses  to  the  questionnaires  on 
observations  that  were  made  during  the  last  exercise  of  the 
just-completed  session. 
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DATA  COLLECTION  FORMS 

Inj&msfegE  Sanaa 

fiEiticaX  Behaviors  Form.  A  critical  incident  approach 
(Flanagan,  1954;  Glickman  &  Vallance,  1958)  was  used  to  develop 
the  Critical  Team  Behaviors  Form,  which  was  used  by  the 
instructors  as  a  means  of  identifying  specific  effective  and 
ineffective  behaviors  of  team  members.  A  sample  of  this  form  is 
given  in  Appendix  B.  The  critical  incident  technique  was  adapted 
here  to  take  advantage  of  the  instructors'  expert  knowledge  of 
the  team  member  behaviors  that  are  linked  to  team  success  or 
failure.  Another  purpose  was  also  served  by  involving  the 
instructors  in  the  development  of  this  instrument.  Namely, 
incidents  that  have  high  salience  for  instructors  should  also  be 
expected  to  have  high  face  validity  for  their  cohorts,  thus 
increasing  their  meaningfulness  when  translated  into  research 
findings  and  recommendations  for  operational  application. 

The  first  step  in  the  development  of  the  Critical  Team 
Behaviors  Form  was  to  conduct  semi-structured  interviews  with  the 
NGFS  instructors.  The  instructors  were  asked  to  describe 
previously  observed  incidents  involving  behaviors  which 
differentiate  more  effective  from  less  effective  teams  on  a 
variety  of  performance  dimensions.  Instructors  were  also 
questioned  about  the  importance  of  several  aspects  of  team 
performance  that  were  identified  from  the  literature  review  that 
resulted  in  the  TEAM  model  (Morgan  et  al.,  1986).  Then  they 
were  asked  to  provide  examples  of  effective  and  ineffective 
behaviors  related  to  the  identified  dimensions  of  team 
performance.  In  order  to  ensure  that  no  important  variable  was 
overlooked,  the  results  of  the  interviews  were  supplemented  by 
research  staff  observations  of  teams  undergoing  NGF*  training. 

The  interviews  were  tape  recorded  and  transc^. or-.d.  a  list  of 
critical  events  was  then  derived  from  the  transcx  ^rionc- ,  and  the 
resulting  items  were  sorted  into  seven  a  priori  dimensions.  In 
order  to  ensure  that  the  broadest  possible  range  of  behaviors 
were  available  to  represent  each  dimension,  additional 
interviews  were  conducted.  These  interviews  were  also  tape 
recorded,  transcribed,  and  content  analyzed.  In  the  final 
categorization,  over  ninety  critical  incidents  were  sorted  into 
the  following  dimensions:  communication,  adaptability, 
cooperation,  acceptance  of  suggestions  or  criticism,  giving 
suggestions  or  criticism,  team  spirit  and  morale,  and 
coordination . 

In  the  initial  draft  of  the  Critical  .Team  Behaviors  Form, 
effective  critical  behaviors  were  separated  from  those  that  were 
classified  as  ineffective  in  their  consequences  for  team  training 
and  performance.  Each  page  of  the  form  contained  either  effective 
or  ineffective  behaviors  for  a  given  dimension.  Instructors  were 
asked  to  place  an  "X"  in  the  box  under  the  position  of  each  of 
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the  members  involved  in  an  observed  critical  incident#  denoting 
the  initiator  of  the  behavior  by  circling  the  appropriate  "X". 

This  initial  version  of  the  Critical  Team  Behaviors  Form 
was  pilot  tested  through  an  iterative  process  that  produced 
several  minor  modifications.  After  the  instructors  had  used  the 
form  at  least  once#  they  were  asked  to  identify  behaviors  which 
they  considered  to  be  unimportant  or  which  occurred  only 
infrequently  during  training.  As  a  result  of  these  efforts,  the 
form  was  revised  so  as  to  present  both  effective  and  ineffective 
behaviors  for  a  given  dimension  in  random  order  on  the  same  page, 
thereby  reducing  the  form  to  about  half  of  its  original  length. 
Finally,  a  page  entitled  "Infrequent  Incidents”  was  added  in 
order  to  allow  instructors  to  report  behaviors  which  occurred 
only  rarely,  but  which  were  considered  to  be  important  when  they 
did  occur.  After  becoming  familiar  with  this  list  of  infrequent 
behaviors,  the  instructors  were  only  required  to  refer  to  this 
page  on  the  rare  occasions  when  one  of  these  behaviors  did  occur. 

From  discussions  with  the  instructors,  it  was  learned  that 
certain  behaviors  were  thought  to  have  greater  impact  than 
others.  Consequently,  an  "impact”  column  was  added  and 
instructors  were  required  to  rate  the  importance  of  each 
behavior  on  a  three-point  scale.  Also,  at  the  suggestion  of 
the  instructors,  a  "frequency  of  occurrence"  column  was  added  to 
the  Communication  and  Cooperation  dimensions  because  behaviors 
within  these  dimensions  could  occur  more  than  once  within  a  given 
exercise. 

The  Critical  Team  Behaviors  Form  is  currently  10  pages  in 
length  and  it  contains  a  total  of  68  behavioral  items  (see 
Appendix  B) .  It  requires  up  to  45  minutes  to  complete.  The 
cover  sheet  contains  the  Instructions  and  a  table  of  contents,  as 
well  as  questions  regarding  the  ship,  the  training  session  (day 
of  training,  morning  or  afternoon) ,  and  the  training  exercise 
just  completed.  Each  of  seven  pages  contains  a  single  dimension 
with  a  list  of  effective  and  ineffective  critical  team  behaviors. 
Another  page  contains  the  list  of  infrequent  behaviors,  and  the 
last  page  provides  space  for  listing  important  happenings 
that  are  not  covered  in  the  preceding  sections. 

Overall  Team  Performance  Form.  The  second  form  requires  the 
instructor  to  provide  an  overall  appraisal  of  the  performance  of 
the  team.  At  the  end  of  each  morning  and  afternoon  session, 
instructors  were  asked  to  rate  the  team's  performance  on  a  five 
point  scale  ranging  from  1  (very  good)  to  5  (very  poor) . 
Descriptive  anchors  were  provided  for  points  1,  3,  and  5. 
Additionally,  instructors  were  asked  to  list  three  team 
strengths  and  weaknesses  and  to  identify  the  training  session 
just  completed.  The  space  for  the  strengths  and  weaknesses  gave 
the  instructors  a  chance  to  supply  any  additional  input  which 
they  considered  to  be  relevant.  The  form  required  about  five 
minutes  to  complete.  A  copy  of  this  form  is  found  in  Appendix  c. 
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Individual  Eerformange  summary.  The  third  form  completed  by 
the  instructors  required  an  appraisal  of  the  performances  of 
the  individual  team  members.  Ratings  were  made  on  a  five  point 
scale  ranging  from  1  (very  good)  to  5  (very  poor)  to  indicate  the 
performance  effectiveness  of  each  team  member  during  each  phase 
of  training.  Instructors  were  also  asked  to  indicate  the  current 
session  of  observation.  This  form  required  about  three  minutes 
for  the  instructor  to  complete.  A  copy  of  the  form  is  found  in 
Appendix  D. 


glo  individual  Eerfsimaasg  Svumary 

The  team's  Gunnery  Liaison  Officer  (GLO)  was  also  required 
to  rate  the  effectiveness  of  the  performances  of  individual  team 
members.  The  form  used  for  this  purpose  is  identical  to  the 
Individual  Performance  Summary  completed  by  the  instructor, 
except  that  the  GLO  was  not  required  to  evaluate  his  own 
performance.  A  copy  of  this  form  is  found  in  Appendix  E. 


Team  Member  Forms 

Trainee  Self -Report  Questionnaire.  Based  on  the  work  of 
James,  Gustafson,  and  Sells  (1985),  a  23-item  self-report 
questionnaire  was  developed  to  measure  the  changing 
perceptions  of  trainees  regarding  individual  and  team  abilities, 
motivation,  and  expertise.  The  questionnaire  was  designed  to 
address  the  dimensions  that  relate  to  individual  skills  as 
well  as  those  that  relate  to  the  team  as  a  whole.  The 
dimensions  associated  with  individual  skills  include:  (1) 
knowledge  of  duties,  (2)  motivation,  (3)  role  clarity,  and  (4) 
experience  and  training.  Dimensions  dealing  with  teamwork 
include:  (1)  communication,  (2)  cooperation  and  coordination, 
(3)  experience  and  prior  training,  and  (4)  power  relationships. 
Items  were  adapted  for  both  of  these  categories  and  then  pilot- 
tested  for  readability  and  redundancy.  No  major  problem  was 
identified  with  regard  to  readability.  However,  based  on  some 
complaints  concerning  redundancy,  several  items  were  reworded  to 
produce  the  final  version  of  the  form. 

The  Trainee  Self-Report  Questionnaire  was  administered  after 
each  morning  and  afternoon  session  beginning  with  Monday 
afternoon  and  concluding  when  the  training  was  completed.  Team 
members  were  requested  to  identify  their  position  on  the  team 
and  to  respond  to  the  questions  in  terms  of  observations  made 
during  the  last  exercise  of  the  just-completed  training  session. 
Twenty-one  of  the  questions  used  a  5-point  scale  with 
corresponding  descriptive  anchors  that  ranged  from  "strongly 
disagree"  to  "strongly  agree."  The  final  two  items  on  the  form 
requested  each  team  member  to  list  the  positions  of  the  "most 
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valuable"  and  "least  valuable"  players  on  the  team.  The  team 
positions  were  listed  randomly  above  the  items  and  assigned  a 
corresponding  number  which  was  to  be  recorded  in  the  space 
provided  after  each  item.  Between  five  and  ten  minutes  were 
required  for  the  team  members  to  complete  this  form.  A  copy  of 
this  questionnaire  is  given  in  Appendix  F. 

Team  Demographics  Form.  This  form  was  developed  to  gather 
information  regarding  overall  Navy  and  NGFS  experience  of  the 
team  members.  The  form  was  pilot  tested  and  revised  three  times 
in  order  to  achieve  economy  of  administration  and  consistency  of 
answers  (e.g.,  the  term  "rate"  often  was  confused  with  "rank"). 
The  final  version  of  the  form,  which  was  administered  during  the 
afternoon  of  the  first  day  of  training,  required  between  five  and 
ten  minutes  to  complete.  This  form  is  reproduced  in  Appendix 
G* 


DATA  COLLECTION  PROCEDURES 

The  typical  training  scenario  required  between  four  and  five 
days,  depending  upon  the  competencies  and  motivations  of  the 
team.  Data  collection  began  on  Monday  afternoon  and  was 
completed  immediately  after  the  team  passed  its  final 
examination,  usually  on  Thursday  afternoon  or  Friday  morning. 
Except  for  the  Demographics  Form,  which  was  completed  only  once 
on  Monday  afternoon,  all  forms  were  administered  at  the  end  of 
each  morning  and  afternoon  training  session;  administration  was 
scheduled  to  follow  the  sessions  so  as  not  to  interfere  with  the 
normal  flow  of  training.  This  schedule  allowed  data  to  be 
collected  at  least  once  during  each  of  the  five  simulator  phases 
of  training.  However,  since  a  greater  amount  of  training  time 
was  usually  allotted  to  the  pre-midterm  and  post-midterm  phases, 
data  were  often  collected  more  than  once  during  these  phases. 

Teams  reported  for  training  on  Monday  morning  of  a  given 
week.  The  pre-exercise  phase  of  training  began  with  a  classroom 
briefing  concerning  the  purposes  and  terminology  of  NGFS 
training.  This  was  followed  by  a  film,  which  demonstrated  the 
types  of  missions  served  by  gunfire  support  teams,  and  a  tour  of 
the  NGFS  simulator  facilities  in  which  the  training  exercises 
were  subsequently  performed.  As  part  of  the  overall  briefing, 
instructors  informed  the  trainees  about  the  nature  and  purposes 
of  the  TEAM  data  collection  effort. 

On  Monday  afternoon,  the  trainees  completed  the  basic 
missions  phase  of  training,  which  began  with  a  lecture  concerning 
the  objectives  of  the  exercises  that  were  to  be  practiced  during 
these  basic  missions.  They  were  then  provided  approximately 
three  and  a  half  to  four  hours  of  training  in  performing  the 
basic  gunfire  support  missions.  At  the  end  of  this  session,  a 
member  of  the  TEAM  research  staff  gave  the  trainees  a  short 
briefing  concerning  the  nature  of  the  data  collection 
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instruments,  the  data  collection  schedule,  and  the  procedures  to 
he  followed  during  each  data  collection  period.  Complete 
instructions  were  given,  all  questions  were  answered,  and  the 
first  set  of  forms  was  administered.  A  member  of  the  TEAM  staff 
distributed  the  appropriate  forms  to  the  instructor  and  trainees, 
and  collected  the  forms  as  they  were  completed. 

As  training  progressed  through  the  remainder  of  the  week, 
TEAM  staff  members  coordinated  their  efforts  with  those  of  the 
instructors  so  that  a  member  of  the  research  staff  could  be 
available  at  the  end  of  each  morning  and  afternoon  session  in 
order  to  administer  the  various  data  collection  instruments. 
Each  administration  required  approximately  10  minutes  of  the  team 
members'  time  and  45  to  50  minutes  of  the  instructor's  time. 
Following  data  collection  after  the  last  session  (at  the  end  of 
the  final  examination  phase  of  training) ,  the  instructors  and 
trainees  were  thanked  for  their  participation  in  the  data 
collection  efforts  and  all  of  their  final  questions  concerning 
the  investigation  were  answered. 
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RESULTS 


OVERVIEW 

The  six  data  collection  instruments  provided  responses  for  a 
total  of  117  individuals  from  the  13  teams  that  completed 
training  during  this  investigation.  These  data  will  be  reported 
in  the  following  sections,  which  separately  present  findings 
based  on  data  from  the  Demographics  Form,  performance  indices 
provided  by  the  graded  training  exercises  and  performance  rating 
scales,  the  Critical  Team  Behaviors  Form,  and  the  Trainee  Self- 
Report  Questionnaire.  In  each  case,  the  reported  analyses  will 
focus  on  two  major  issues:  (a)  providing  a  description  of  the 
evolution  and  maturation  of  teams,  and  (b)  identifying  the 
characteristic  differences  between  the  highest  performing  and 
lowest  performing  teams.  Evolution  and  maturation  effects  will 
be  examined  in  terms  of  the  changes  in  teams'  performances, 
behavior  patterns,  and  questionnaire  responses  over  the  phases  of 
training.  Discrimination  effects  will  be  examined  in  tarms  of 
similar  differences  among  the  teams.  For  the  purposes  of  these 
analyses,  data  from  each  morning  and  afternoon  data  collection 
session  have  been  assigned  to  represent  the  training  phase  that 
was  in  progress  at  the  time  of  the  data  collection.  When  two  or 
more  data  collection  sessions  occurred  during  a  given  phase  of 
training,  data  were  combined  so  as  to  provide  a  single  data  point 
for  that  phase. 


DEMOGRAPHICS  OF  THE  SAMPLE 

Although  the  guidelines  for  NGFS  training  prescribe  that  CIC 
teams  will  consist  of  eight  team  members,  the  teams  in  the 
current  sample  ranged  in  size  from  seven  to  twelve,  with  an 
average  of  nine  team  members.  However,  data  from  the 
Demographics  Form  indicate  that  the  individuals  in  this  sample 
were  relatively  inexperienced  in  NGFS  performance.  This  is 
illustrated  in  Table  1,  which  provides  summary  data  from  five  of 
the  twelve  questions  on  the  Demographics  Form.  Specifically,  the 
table  describes  the  sample  in  terms  of  the  team  members'  rank, 
years  in  the  Navy,  years  of  NGFS  experience,  prior  schoolhouse 
training,  and  frequency  of  NGFS  practice. 

These  data  indicate  that  75%  of  rhe  sample  were  enlisted 
men,  and  nearly  63%  held  the  rank  of  E5  or  lower.  Nearly  35%  of 
the  sample  reported  having  less  than  two  years  of  service  in  the 
Navy  and  almost  72%  had  less  than  five  years  in  the  Navy. 
Similarly,  approximately  64%  of  the  trainees  had  less  than  one 
year  of  NGFS  experience;  only  6%  reported  having  more  than  two 
years  of  such  experience.  Furthermore,  only  about  one-half  of 
the  sample  had  ever  been  exposed  to  formal  NGFS  training,  and 
approximately  32%  had  never  practiced  NGFS  procedures  aboard 
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Table  1 

Demographic  Characteristics  of  the  NGFS  Sample 


Rank  in  the  Navy 

E2  E3 

E4  E5  E6 

E7-9  Ol  02 

03 

NA 

Frequency 

1  15 

30  27  11 

3  8  6 

9 

6 

%  of  Total 

0.8  12. 

9  25.9  23.3  9.5 

2.6  6.9  5. 

2  7.8 

5.2 

Years  in  the 

i  Navy 

Less 

than  1 

1-2  3-4  5-6 

7-8  9-10 

11  or 

more 

Frequency 

5 

35  43  12 

4  11 

6 

%  of  Total 

4.3 

30.2  37.1  10.3 

3.4  9.5 

5. 

2 

Years  in  NGFS  Training 

Less  than 

i  1  1-2 

3-5 

NA 

Frequency 

74 

32 

7 

3 

%  of  Total 

63.8 

27.6 

6.0 

2.6 

Prior  NGFS  Training 

No 

Yes 

Frequency 

56 

60 

%  of  Total 

48.3 

51.7 

Frequency  of 

NGFS  Practice 

Never 

Twice  a  Month, 

More  than  Twice 

or  less 

a 

month 

Frequency 

37 

41 

38 

%  of  Total 

31.9 

35.3 

32.8 
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ship.  The  sample  was  divided  roughly  into  thirds  on  the  basis  of 
the  frequency  of  NGPS  practice/  with  only  about  33%  practicing 
more  than  twice  per  month.  Overall,  these  data  indicate  that  the 
typical  (median)  team  member  in  the  current  sample  was  an 
enlisted  sailor  with  a  rank  of  E5,  approximately  three  years  of 
Navy  experience,  less  than  one  year  of  assignment  to  NGFS  duties, 
no  formal  NGFS  training,  and  relatively  little  prior  practice  of 
NGFS  duties.  These  data  further  suggest  that  a  considerable 
percentage  of  the  team  members  assigned  to  training  at  the  NGFS 
school  enter  training  with  less  than  adequate  NGFS  skills  and 
background. 


INDICES  OF  TEAM  PERFORMANCE 
Graded  Exercises 

The  relative  success  of  teams  in  training  was  formally 
assessed  by  the  instructors  during  the  midterm  and  final  phases 
of  training.  These  phases  consist  of  standard  simulator 
exercises  in  which  overall  team  performance  is  graded  by  the 
instructor  according  to  a  prescribed  scoring  protocol  (a  summary 
of  the  scoring  procedure  is  given  in  Appendix  H)  .  The  maximum 
possible  score  on  these  exercises  is  100  points,  and  the  minimum 
passing  score  is  70.  Thus,  the  midterm  and  final  scores  provide 
relatively  objective  assessments  of  team  performance  near  the 
midpoint  and  at  the  end  of  training.  These  scores  are  reported 
in  Table  2  for  each  of  the  13  teams. 

These  data  indicate  that  the  teams'  performance  scores 
generally  increased  from  the  midterm  (M  -  78.62)  to  the  final  (M 
■  86.92)  phases  of  training  (£(24)  -  2.22,  p<.01).  This  is  taken 
as  evidence  of  the  general  efficacy  of  the  training  experience. 
It  should  be  noted,  however,  that  for  a  given  team  the  midterm 
and  final  performances  were  evaluated  by  the  same  instructor. 
Thus  it  is  possible  that  the  indicated  increase  in  scores  is  the 
result  of  an  instructor  bias  toward  showing  the  improvement  in 
performance.  This  seems  to  be  refuted  in  the  current  case  by  the 
fact  that  the  midterm  and  final  scores  were  not  significantly 
correlated  (£(13)  ■  .46,  p>.05).  As  shown  in  Table  2,  almost 
the  complete  range  of  possible  scores  was  employed  in  both  data 
sets,  and  for  three  teams  the  scores  actually  decreased  from  the 
midterm  to  the  final.  Thus,  there  is  no  evidence  of  bias  in  the 
reported  scores.  It  should  be  noted,  however  that  the  average 
final  examination  score  of  86.92  leaves  considerable  room  for 
improving  the  overall  impact  of  training  on  the  final 
performances  of  NGFS  teams. 

The  midterm  and  final  examination  scores  were  correlated 
with  scores  from  the  qualification  exercises  on  the  live-fire 
range.  At  the  completion  of  the  current  research  eight  of  the 
thirteen  sample  teams  had  completed  their  live  firing  exercises. 
For  this  subset  of  eight  teams,  performances  on  the  final 
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Table  2 

Midterm  and  Final  Examination  Scores  for  Each  Team 


Ship 

Number 

Midterm 

Score 

Final 

Score 

1 

86 

90 

2 

97 

87 

3 

83 

98 

4 

70 

96 

5 

70 

78 

6 

92 

94 

7 

69 

70 

8 

78 

88 

9 

70 

78 

10 

70 

80 

11 

70 

89 

12 

95 

91 

13 

72 

91 

Average  Score 

78.62 

86.92 

examination  were  significantly  correlated  with  the  live-fire 
qualification  scores  (£(8)  -  .75,  pc. 05).  This  suggests  that  by 
the  time  of  the  final  phase  of  training,  teams  have  acquired 
behaviors  that  are  predictive  of  later  performance  during  live 
firing.  It  should  be  noted,  however,  that  in  the  current  sub¬ 
sample  of  eight  ships,  the  midterm  exam  scores  were  not 
significantly  correlated  with  the  performance  scores  from  the 
range  (£(8)  =.47,  n>,05).  This  suggests  that  the  behaviors  that 
are  critical  for  successful  performance  on  the  range  have  not 
been  acquired  by  the  midterm  phase  of  training. 

Having  established  that  the  teams'  final  examination  scores 
are  a  valid  representation  of  performance  in  the  field,  the  final 
scores  were  used  as  a  criterion  for  dividing  the  teams  into 
groups  containing  the  more  effective,  medium  effective,  and  less 
effective  teams.  These  divisions  were  then  used  in  order  to 
analyze  subsequent  data  sets  for  discrimination  effects.  The 
three  groups  of  final  examination  scores  are  presented  in  order 
of  performance  proficiency  in  Table  3. 

Based  on  the  natural  groupings  of  the  13  scores,  three  teams 
(3,  4,  and  6)  with  scores  of  94  to  98  were  placed  into  the  more 
effective  category,  and  six  teams  (l,  2,  8,  11,  12,  and  13)  with 
scores  of  87  to  91  were  categorized  in  the  effective  set.  The 
remaining  four  teams  (5,  7,  9,  and  10)  with  scores  of  70  to  80 
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comprise  the  less  effective  set.  In  terms  of  these  scores,  the 
final  performances  of  the  effective  teams  differed  significantly 
from  both  the  more  effective  (£(7)  ■  5.53,  pc.OOl)  and  the  less 
effective  (£(8)  »  6.59,  pc. 001)  teams.  It  should  be  noted  that 
all  of  the  teams  in  the  sample  successfully  completed  training. 
Therefore,  some  restriction  in  the  range  of  scores  is  imposed  on 
the  data  presented  in  the  sections  to  follow. 


Table  3 


Final 

Examination 

Scores  for  NGFS 

Teams 

Performance 

Ship 

Final 

Category 

Category 

Number 

Score 

Mean 

More  Effective 

3 

93 

4 

96 

96.0 

6 

94 

Effective 

12 

91 

13 

91 

1 

90 

89.3 

11 

89 

8 

88 

2 

87 

Less  Effective 

10 

80 

5 

78 

9 

78 

76.5 

7 

70 

Performance  Ratings 

The  performances  of  the  teams  were  also  assessed  by  (a) 
having  the  instructor  of  each  team  to  rate  the  overall 
performance  of  the  team  (using  the  Overall  Team  Performance  Form) 
and  the  individual  performance  levels  of  each  team  member  (using 
the  Instructor  Individual  Performance  Summary  Form) ,  and  (b) 
requiring  the  team's  leader  (the  Gunnery  Liaison  Officer;  GLO)  to 
rate  the  performance  of  all  team  members  except  himself  (using 
the  GLO  Individual  Performance  Summary  Form) .  The  individual 
performance  ratings  were  averaged  across  team  members  within  each 
phase  of  training  in  order  to  produce  a  mean  performance  rating 
for  each  team  in  each  phase.  These  data  are  compared  to  the 
instructors'  overall  performance  ratings  in  Figure  5.  In  this 
and  subsequent  graphs  of  rating  scale  data,  the  obtained  values 
have  been  subtracted  from  5.0  in  order  to  reverse  the  scale.  Thus 
4.0  represents  the  highest  performance  and  1.0  the  lowest. 
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The  data  in  Figure  5  indicate  that  the  ratings  made  by  both 
the  instructors  and  GLOs  generally  increased  across  the  phases  of 
training.  That  is,  both  individual  and  overall  team  performances 
were  judged  to  have  improved.  This  finding  provides  further 
evidence  of  the  general  efficacy  of  the  training  exercises.  The 
data  of  Figure  5  also  indicate  that  the  GLO  tended  to  rate  the 
performances  of  his  team  members  higher  than  did  the  instructors. 
Correlations  among  the  ratings  within  phases  indicate  that  the 
GLO's  ratings  of  individual  performances  were  significantly 
correlated  with  the  instructors'  ratings  of  individual  and  team 
performances  only  during  the  midterm  (£(13)  *  *61*  R<.05  and 
r(13)  -  .59,  p< . 05 ,  respectively)  and  final  phases  of  training 
(£(13)  ■  .61,  pc. 05  and  £(13)  «  .57,  pc. 05,  respectively).  On  the 
other  hand,  the  instructors'  ratings  of  individual  performance 
were  significantly  correlated  with  their  overall  performance 
ratings  within  each  phase,  similarly,  correlations  across  phases 
indicate  that  pairs  of  ratings  by  the  instructor  were  almost 
always  significantly  correlated.  However,  ratings  by  the  GLo 
were  almost  never  significantly  correlated  with  either  of  the 
instructors'  ratings.  Thus,  as  would  be  expected,  the  two  sets 
of  ratings  by  the  instructor  are  more  consistently  related  to 
each  other  than  to  the  ratings  by  the  GLO.  Only  when  objective 
test  scores  were  available,  in  the  midterm  and  final  phases,  were 
the  three  sets  of  ratings  in  relative  agreement. 

Figure  5  also  shows  that  the  instructors'  overall  ratings  of 
the  teams  were  the  lowest  of  the  ratings.  This  suggests  that 
individual  team  members  were  seen  as  performing  better  than  the 
team  as  a  whole.  However,  this  finding  probably  resulted  from 
the  fact  that  the  instructor  was  more  reluctant  to  assign  poor 
ratings  to  a  specific  individual  (where  direct  responsibility  and 
accountability  accrue)  than  to  the  overall  team  (where 
responsibility  and  accountability  are  dispersed) . 

When  averaged  across  the  training  phases  for  each  team,  the 
instructor  and  GLO  ratings  were  found  to  be  significantly 
intercorrelated.  These  correlations  were  computed  by  averaging 
each  set  of  ratings  across  phases  and  intercorrelating  (across 
the  13  teams)  the  resulting  mean  individual  and  overall  team 
performance  ratings.  As  shown  in  Table  4,  each  set  of  mean 
ratings  was  significantly  related  to  the  other  two  sets  of 
ratings.  of  course,  the  instructors'  overall  and  individual 
performance  ratings  were  most  highly  correlated  (£(13)  ■  .91, 
pc. 05) .  This  pattern  of  intercorrelations  indicates  that  a  team's 
average  evaluation  was  relatively  the  same  regardless  of  the 
source  of  the  evaluation,  and  this  provides  some  evidence  of 
convergent  validity  for  the  various  scales.  It  should  be  noted 
that  only  the  instructors'  ratings  were  significantly  related  to 
mean  scores  on  the  midterm  examination  (£(13)  =  .48,  pc, 05  and 
£(13)  -  .48,  pc. 05  for  the  ratings  of  individual  performance  and 
team  performance,  respectively)  ,  and  that  none  of  the  mean 
ratings  were  significantly  correlated  with  performance  during  the 
final  exam  phase  of  training. 
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Figure  5.  Mean  ratings  by  the  Gunnery  Liaison  Officer 
and  Instructor  of  individual  team-member 
performances  and  mean  Instructor  ratings 
of  overall  team  performance  for  each  phase 
of  training. 
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Table  4 

Intercorrelations  of  Mean  Performance  Ratings  by  the 
Instructor  and  the  Gunnery  Liaison  Officer  (GLO) 


Rater/  Instructor  Instructor/Overall 

Rating  Individual  Performance  Team  Performance 


Instructor/Individual 

Performance  -  .91* 


GLO/Individual 

Performance  .56*  .49* 


n  *  13  for  both  groups. 
*  p.c.05. 


The  performance  ratings  for  the  more  effective,  effective, 
and  less  effective  groups  of  teams  are  shown  in  Figure  6.  In 
general,  the  ratings  are  somewhat  higher  for  the  more  effective 
teams,  but  these  differences  are  not  large.  The  major 
differences  in  these  patterns  of  ratings  are  seen  in  the  facts 
that  (a)  the  instructors  rated  the  midterm  and  final  performances 
of  the  more  effective  teams  relatively  highly,  suggesting  that 
they  felt  that  these  teams  (in  contrast  to  the  other  two  groups) 
performed  better  in  these  ''evaluation”  phases  than  in  the  other 
phases;  and  (b)  the  instructors  rated  the  overall  performances 
of  the  most  effective  teams  higher,  or  nearly  as  high,  as  the 
average  individual  performances  of  the  members  ol  these  teams. 
This  latter  point  clearly  suggests  that  the  teamwork  skills  of 
these  teams  were  better  that  those  of  the  other  groups  of  teams — 
so  much  better,  in  fact,  that  in  three  phases  the  overall 
performance  of  these  teams  was  judged  to  be  better  than  the 
performance  of  the  average  individual  team  member. 

In  order  to  examine  the  relative  performances  of  individual 
team  members,  each  team  member's  performance  ratings  by  the 
instructor  and  the  GLO  were  averaged  across  the  phases  of 
training.  These  computations  yielded  the  means  and  standard 
deviations  presented  in  Table  5,  which  also  presents  the 
correlations  between  the  instructor  and  GLO  ratings  of  each 
individual's  performance.  These  data  indicate  that  when  averaged 
across  phases,  the  ratings  (particularly  those  of  the  instructor) 
provide  relatively  small  discriminations  among  the  team  members. 
Nevertheless,  it  is  clear  that  both  the  instructor  and  the  GLO 
placed  the  AGLO  and  CIC  Supervisor  positions  among  the  highest- 
rated  performers.  Table  5  also  indicates  that  the  instructor  and 
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Figure  6.  Mean  ratings  of  individual  (rated  separately 
by  the  Gunnery  Liaison  Officer  and  Instructor) 
and  team  (rated  by  the  Instructor)  performance 
by  phase  for  the  more  effective ,  effective, 
and  less  effective  teams. 


Technical  Report  87-016 


GLO  ratings  were  significantly  correlated  for  four  of  the  team 
member  positions.  This  means  that  the  performances  of  the 
individuals  in  these  positions  were  ranked  in  relatively  the  same 
order  across  the  13  teams  by  both  the  instructors  and  GLOs. 


Table  5 

Means,  Standard  Deviations,  a*nd  Correlations  of  Individual 
Performance  Ratings  by  the  Instructor  and  the  GLO 


Source  of  Ratings 

Position 

Instructor 

GLO 

£ 

Means 

(S.D.) 

Means 

(S.D.) 

GIO 

2.75 

(0.83) 

AGLO 

2.99 

(0.84) 

3.34 

(0.63)  ■; 

0.64* 

CIC  Supervisor 

2.96 

(1.04) 

3.17 

(0.82) 

0.67* 

NAV  Plotter 

2.89 

(1.06) 

3.03 

(0.74) 

0.48* 

Target  Plotter 

2.92 

(0.99) 

2.96 

(0.87) 

0.43 

R/T  Talker 

2.69 

(1.03) 

3.20 

(0.64) 

0.31 

R/T  Recorder 

2.67 

(1.01) 

3.12 

(0.71) 

0.64* 

NAV  Recorder 

2.85 

(0.85) 

3.08 

(0.71) 

0.37 

n  -  13 
*  pc. 05 


The  relative  performances  of  the  individuals  who  occupied 
the  positions  of  the  three  major  team  leaders  were  examined  by 
computing  the  instructors'  average  individual  performance  ratings 
for  the  GLO,  AGLO,  and  CIC  Supervisor  for  each  phase  of 
training.  These  data  are  shown  in  Figure  7  for  the  three  more 
effective  and  four  less  effective  teams.  The  ratings  are 
somewhat  higher  for  the  more  effective  teams,  and  the  patterns  of 
ratings  ore  different  for  the  two  groups.  For  the  more  effective 
teams,  the  GLO  and  AGLO  are  rated  lower  than  the  CIC  Supervisor 
during  basics,  but  their  ratings  rapidly  improve  to  the  point 
where  their  performances  are  rated  considerably  higher  than  that 
of  the  CIC  Supervisor  during  the  final  phase.  In  the  less 
effective  teams,  the  ratings  of  the  GLO  are  quite  low 
(particularly  during  the  three  middle  phases  of  training) ,  and 
the  ratings  of  CIC  Supervisor  improve  during  the  course  of 
training,  but  those  of  the  GLO  and  AGLO  do  not.  Thus,  it  appears 
that  in  the  more  effective  teams  the  designated  team  leader  (the 
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Figure  7.  Means  of  the  Instructors'  ratings  of  the 
individual  performances  of  the  Gunnery 
Liaison  Officer,  Assistant  Gunnery  Officer 
and  CIC  Supervisor  by  phase  of  training 
for  the  more  and  less  effective  teams. 
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GLO)  adjusts  to  his  role  and  provides  effective  leadership, 
whereas  in  the  less  effective  teams  the  GLO  fails  to  emerge  as  an 
effective  leader.  It  seems  that  improvements  in  performance  on 
•the-peet—of— the-less_-effective..  teams  must,  be  attributed  in  large 
part  to  the  contributions  of  the  CIC  Supervisor. 


CRITICAL  TEAM  BEHAVIORS 

In  completing  the  Critical  Team  Behaviors  Form,  the  NGFS 
instructors  reviewed  lists  of  effective  and  ineffective  behaviors 
for  each  of  seven  behavioral  dimensions  (communication, 
cooperation,  team  spirit  and  morale,  giving  suggestions  or 
criticisms,  coordination,  accepting  suggestions  or  criticisms, 
and  adaptability)  and  for  a  category  of  infrequently  occurring 
behaviors  (see  Appendix  B  for  a  copy  of  the  Critical  Team 
Behaviors  Form) .  They  marked  the  form  so  as  to  indicate  which  of 
the  listed  behaviors  occurred  during  a  just-completed  training 
session.  They  also  identified  the  initiator (s)  and  the 
recipient (s)  of  each  reported  behavior.  Further  discussion  of 
the  Critical  Team  Behaviors  Form  can  be  found  in  the  Methods 
section  and  in  Morgan  et  al.  (1986). 

Using  the  Critical  Team  Behaviors  Form,  the  instructors 
recorded  a  total  of  1102  critical  behaviors  for  the  13  teams.  Of 
these,  832  (75%)  were  effective  behaviors  and  270  (25%)  were 
ineffective  behaviors.  It  should  be  noted  that  the  Critical  Team 
Behaviors  Form  actually  lists  a  total  of  68  behaviors,  of  which 
34  (50%)  are  effective  and  34  (50%)  are  ineffective  behaviors. 
Thus,  there  seems  to  be  a  decided  tendency  for  instructors  to 
record  relatively  more  effective  than  ineffective  critical 
events.  Whether  or  not  this  literally  mirrors  actual  events 
cannot  be  said  at  this  time.  The  distributions  of  reports  of 
effective  and  ineffective  behaviors  across  and  within  phases  are 
given  in  Table  6.  These  data  indicate  that  the  greatest 
proportion  of  behaviors  occurred  in  the  pre-midterm  phase,  but 
this  is  to  be  expected  because  a  minimum  of  two  administrations 
of  the  critical  Team  Behaviors  Form  occurred  during  this  phase. 

As  suggested  above,  the  Critical  Team  Behaviors  data  were 
submitted  to  two  lines  of  analysis.  In  the  first,  evolution  and 
maturation  effects  were  tested  by  examining  the  changes  in  team 
behaviors  that  occurred  over  the  phases  of  training.  In  the 
second,  discrimination  effects  were  examined  in  terms  of  the 
behavioral  differences  that  occurred  in  the  more  effective  and 
less  effective  teams.  In  both  of  these  cases,  the  behaviors  of 
the  individuals  in  the  leadership  positions  of  the  teams  (i.e., 
the  GLO,  AGLO,  CIC  supervisor,  and  instructor)  were  specifically 
analyzed  in  order  to  find  the  extent  to  which  the  leadership 
interactions  reported  here  conformed  to  the  expectations  provided 
by  the  Component  Model  of  NGFS  Training  (see  Figure  3) . 
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Table  6 

Distributions  of  Effective  and  Ineffective  Behaviors 
_ _ _ by  Phase  ^ 


Training  Phase 


Distribution 

Basic 

Pre- 

Midterm 

Midterm 

Post- 

Midterm 

Final 

Across  Phases 

rffective 

216 

273 

106 

146 

91 

26.0% 

32.8% 

12.7% 

17.5% 

11.0% 

Ineffective 

73 

107 

33 

37 

20 

27.0% 

39.6% 

12.2% 

13.8% 

7.4% 

Total 

289 

380 

139 

183 

111 

26.2% 

34.5% 

12.6% 

16.6% 

10.1% 

Within  phases 

Effective 

216 

273 

106 

146 

91 

74.7% 

71.8% 

76.3% 

80.0% 

82.0% 

Ineffective 

73 

107 

33 

37 

20 

25.3% 

28.2% 

23.7% 

20.0% 

18.0% 

Total 

289 

380 

139 

183 

111 

100.0% 

100.0% 

100.0% 

100.0% 

100.0% 

fi£  Total  fitehay lara 

Effective 

216 

273 

106 

146 

91 

19.6% 

24.8% 

9.6% 

13.2% 

8.3% 

Ineffective 

73 

107 

33 

37 

20 

6.6% 

9.7% 

3.0% 

3.4% 

1.8% 

Total 

289 

380 

139 

183 

111 

26.2% 

34.5% 

12.6% 

16.6% 

10.1% 

Totals 


832 

100.0% 

270 

100.0% 

1102 

100.0% 


832 

75.5% 

270 

24.5% 

1102 

100.0% 


832 

75.5% 

270 

24.5% 

1102 

100.0% 
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The  overall  distributions  of  critical  behaviors  provide 
insights  concerning  the  development  of  team  behaviors  during 
training.  In  terms  of  relative  frequencies  across  phases  (the 
upper  panels  of  Table  6) ,  the  data  indicate  that  the  fewest 
behaviors  were  reported  in  the  final  phase  and  the  next  fewest  in 
the  midterm  phase.  This  suggests  that  either  (a)  the  evaluation 
exercises  performed  in  these  phases  allowed  relatively  little 
opportunity  for  the  occurrence  of  critical  behaviors,  or  (b)  the 
attention  of  instructors  was  concentrated  upon  scoring  these  test 
exercises,  so  that  they  failed  to  observe  and  report  specific 
behaviors  of  the  participants.  Whatever  the  cause,  the  reporting 
of  effective  and  ineffective  behaviors  seems  to  have  been 
similarly  affected  across  phases. 


As  shown  in  middle  rows  of  Table  6,  the  percentages  of 
behaviors  occurring  within  phases  indicate  that  the  training 
resulted  in  a  general  improvement  in  performance.  The  relative 
percentages  of  effective  behaviors  increased  and  ineffective 
behaviors  decreased  substantially  during  the  last  three  phases  as 
compared  to  the  first  two  phases.  The  parallel  between  this 
improvement  in  the  relative  proportion  of  effective/ ineffective 
behaviors  and  the  previously  cited  improvement  in  the  indices  of 
team  performance  adds  weight  to  the  inference  that  team 
performance  was  enhanced  because  teams  learned  to  engage  in 
effective  behaviors  more  often,  and  to  correct  or  eliminate 
ineffective  behaviors. 


This  inference  is  illustrated  further  in  Table  7,  which 
compares  the  within-phase  percentages  of  effective  and 
ineffective  behaviors  in  the  first  two  and  the  last  three  phases 
for  each  of  the  dimensions  represented  in  the  critical  Team 
Behaviors  Form.  For  example,  12.5%  of  the  total  number  of 
Communication  behaviors  reported  in  the  Basics  and  Pre-midterm 
phases  were  effective  behaviors  (87.5%  were  ineffective),  while 
16.4%  of  all  behaviors  in  the  midterm,  post-midterm,  and  final 
phases  were  effective  (83.6%  were  ineffective).  Thus, 
performance  in  the  Communication  dimension  improved  in  the  latter 
phases  in  that  a  relatively  greater  percentage  of  reported 
behaviors  were  effective.  (It  should  be  noted  that  nearly  87%  of 
the  behaviors  listed  in  the  Communication  dimension  and  the 
category  of  Infrequent  Events  were  ineffective  behaviors. 
Therefore,  the  percentages  obtained  within  these  categories  are 
to  be  expected.) 

Similar  improvements  in  the  relative  occurrence  of  effective 
behaviors  can  be  seen  within  most  of  the  other  behavioral 
dimensions.  For  example,  for  Accepting  Suggestions,  the  relative 
percentage  of  effective  behaviors  increased  from  70.6%  (29.4% 
ineffective)  prior  to  the  midterm  to  94.7%  (5.3%  ineffective) 
during  and  subsequent  to  the  midterm.  Closer  examination  of  +  e 
responses  in  this  dimension  indicates  that  the  improvement  he  a 
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Table  7 

Within-Phase  Percentages  of  Effective  and  Ineffective 
Behaviors  by  Dimension  for  Combinations 
of  the  First  Two  and  Last  Three  Phases 


Dimension 

Number  (and 
Percentage) 
of  Items  Per 
Dimension 

Phases 

Basic  and 
Pre-Midterm 

Combined 

Midterm,  Post- 
Midterm,  and 
Final 

Communication 

Effective 

1  (14.3%) 

12.5% 

16.4% 

Ineffective 

6  (85.7%) 

87.5% 

83.6% 

Cooperation 

Effective 

6  (75.0%) 

88.6% 

94.7% 

Ineffective 

2  (25.0%) 

11.3% 

5.3% 

Team  Spirit  &  Morale 

Effective 

7  (58.3%) 

97.1% 

92.6% 

Ineffective 

5  (41.7%) 

2.9% 

7.4% 

giving  Suggestions 

Effective 

5  (62.5%) 

82.8% 

88.9% 

Ineffective 

3  (37.5%) 

17.2% 

11.1% 

Accepting  suggestions 

Effective 

2  (33.3%) 

70.6% 

94.7% 

Ineffective 

4  (66.7%) 

29.4% 

5.3% 

Coordination 

Effective 

7  (63.6%) 

79.1% 

86.8% 

Ineffective 

4  (36.4%) 

20.9% 

13.2% 

Adaptability 

Effective 

5  (62.5%) 

81.3% 

81.3% 

Ineffective 

3  (37.5%) 

18.7% 

18.7% 

infreaaant  Evanta 

Effective 

1  (12.5%) 

13.3% 

14.3% 

Ineffective 

7  (87.5%) 

86.7% 

85.7% 

Technical  Report  87-016 


resulted  primarily  from  reports  of  effective  and  more  effective 
team  members  thanking  other  team  members  for  catching  their 
errors . 

Overall,  these  data  indicate  that  the  manifestations  of  good 
teamwork  skills  increased  with  training  in  the  dimensions  of 
Communication,  Cooperation,  Giving  Suggestions,  Accepting 
Suggestions,  and  Coordination.  Such  improvement  is  not  seen, 
however,  in  the  data  for  Team  Spirit  and  Morale,  and  for 
Adaptability,  or  the  category  of  Infrequent  Events.  In  the  cases 
of  the  Adaptability  dimension  and  the  category  of  Infrequent 
Events,  there  was  no  real  change  in  the  relative  percentage  of 
effective  behaviors  across  phases.  However,  in  contrast  to  the 
pattern  exhibited  by  the  other  dimensions,  the  relative 
percentage  of  effective  behaviors  decreased  slightly  across 
phases  in  the  Team  Spirit  and  Morale  dimension.  Closer 
examination  of  the  data  indicates  that  these  percentages  are 
based  on  very  small  numbers  of  behaviors  and  that  they  resulted 
from  the  fact  that  in  the  later  phases  of  training  members  of 
less  effective  teams  argued  among  themselves  and  made  negative 
comments  about  the  team  or  training. 

Another  evolution  and  maturation  effect  was  examined  by 
taking  note  of  the  extent  to  which  the  instructor  and  the  team 
leaders  (i.e.,  the  GLO,  AGLO,  and  CIC  Supervisor)  were  involved 
in  the  activities  of  the  team  during  the  various  phases  of 
training.  These  results  are  particularly  interesting  with 
respect  to  the  involvement  of  the  instructor.  Based  of  the 
Component  Model  of  NGFS  team  development  (see  Figure  3) ,  it  was 
predicted  that  the  instructor  would  be  heavily  involved  in  the 
teams'  activities  during  the  early  phases  of  training  and  that  he 
would  withdraw  himself  from  the  team  in  later  phases  of  training. 
This  prediction  is  addressed  in  Table  8,  which  presents  (for  each 
phase)  the  number  of  recorded  interactions  in  which  the 
instructor  was  the  initiator  or  recipient  of  a  critical  behavior. 

These  frequencies  clearly  show  that  the  instructor  actively 
participated  with  the  team  during  the  basics  and  pre-midterm 
phases,  but  that  his  participation  decreased  dramatically 
following  the  pre-midterm  phase.  These  data  strongly  support  the 
notion  that  most  instruction  takes  place  early  in  training  and 
that  the  team  is  put  "on  its  own"  to  a  large  extent  after  the 
first  two  phases.  This  is  particularly  evidenced  by  the 
frequencies  with  which  the  instructors  initiated  behaviors  toward 
team  members.  As  the  table  shows,  the  instructor  initiated  only 
two  behaviors  during  the  last  three  phases  of  training.  Thus, 
one  index  of  team  development  is  the  extent  to  which  it  continues 
to  be  necessary  for  the  instructor  to  be  actively  involved  in  the 
team's  activities. 
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Table  8 


Number  of  Instructor-Related  Behaviors  by  Phase 


Instructor ' s 
Role 

Phase 

Total 

Basic 

Pre- 

Midterm 

Midterm 

Post- 

Midterm 

Final 

Initiator 

14 

28 

1 

1 

0 

44 

Recipient 

38 

41 

4 

21 

1 

105 

Total 

52 

69 

5 

22 

1 

149 

pjggrlminatign  EEtssfea 

The  Critical  Team  Behaviors  data  were  also  analyzed  to 
uncover  differences  between  the  more  effective  and  the  less 
effective  teams.  These  differences  are  reflected  in  Table  9, 
which  provides  for  each  phase  and  each  dimension  on  the  Critical 
Team  Behaviors  Form  the  number  of  effective  and  ineffective 
behaviors  displayed  by  the  more  effective  and  less  effective 
teams.  It  should  be  noted  that  there  were  three  more  effective 
teams  and  four  less  effective  teams,  however,  because  of 
uncontrollable  circumstances  data  were  collected  for  only  three 
of  the  less  effective  teams  during  the  postmidterm  phase  and  two 
of  them  during  the  final  phase  of  training.  The  means  for  the 
less  effective  teams  (the  rightmost  column)  were  computed  on  the 
basis  of  the  actual  number  of  data  points  available. 

The  totals  in  Table  9  show,  that  the  teams  differed  only 
slightly  in  terms  of  the  number  of  ineffective  behaviors 
(overall,  both  groups  averaged  approximately  25  ineffective 
behaviors  per  team).  The  major  difference  between  the  effective 
and  ineffective  teams  is  in  the  proportion  of  effective  behaviors 
exhibited;  the  more  effective  teams  averaged  66%  more  effective 
behaviors  (93  per  team)  than  the  ineffective  teams  (about  56  per 
team).  Thus,  for  the  more  effective  teams,  approximately  79%  of 
the  reported  behaviors  were  effective  (21%  were  ineffective) , 
whereas  for  the  less  effective  teams,  only  about  68%  of  the  total 
behaviors  were  effective  behaviors.  This  suggests  that 
successful  team  performance  is  more  dependent  on  learning  which 
effective  behaviors  to  perform  than  which  ineffective  behaviors 
to  avoid. 

These  data  must  be  interpreted  in  light  of  the  reported 
differences  in  instructor's  perceptions  of  and  responses  to  teams 
with  "good"  attitudes  and  teams  with  "poor"  attitudes,  as 
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Table  9 

Effective  and  Ineffective  Behaviors  by  Dimension  and  Ehase 
for  the  More  Effective  and  Less  Effective  Teams 


Bass 


Pre-Mid  Mid  Post-Mid 


Dimension 

Basics 

term 

term 

term 

Final 

Total 

Mean 

ME  LE 

me  Se 

ME  LE 

ME  IE 

ME  IE 

ME  IE 

ME 

IE 

Number  of  teams 

3  4 

3  4 

3  4 

3  3 

3  2 

15  17 

3.00 

3.40 

Octnmunicatlon 


Effective 

3 

0 

4 

2 

1 

1 

3 

1 

1 

0 

12 

4 

4.00 

1.18 

Ineffective 

9 

9 

16 

16 

7 

6 

2 

6 

3 

2 

37 

39 

2.33 

11.47 

^aSESEffec±ive 

12 

15 

22 

17 

9 

3 

9 

14 

7 

2 

59 

51 

19.67 

15.00 

Ineffective 

2 

3 

2 

1 

0 

2 

0 

1 

0 

0 

4 

7 

1.33 

2.09 

Team  Spirit  &  MgsOs 
Effective  11 

14 

23 

13 

7 

2 

8 

3 

7 

5 

56 

37 

18.67 

10.88 

Ineffective 

0 

0 

1 

0 

0 

2 

0 

0 

0 

2 

1 

4 

0.33 

1.18 

giving  suggestions 

Effective 

7 

6 

13 

8 

4 

2 

6 

5 

3 

0 

33 

21 

11.00 

6.18 

Ineffective 

2 

0 

2 

3 

0 

1 

0 

0 

0 

0 

4 

4 

1.33 

1.18 

Accepting  Suggestions 
Effective  4 

3 

6 

2 

2 

0 

2 

0 

4 

0 

18 

5 

6.00 

1.47 

Ineffective 

1 

0 

1 

3 

0 

0 

0 

0 

0 

0 

2 

3 

0.67 

0.88 

Coordination 

Effective 

12 

19 

26 

12 

10 

2 

9 

11 

10 

5 

67 

49 

22.33 

14.41 

Ineffective 

3 

4 

4 

7 

0 

1 

1 

3 

1 

1 

9 

16 

3.00 

4.71 

Adaptability 

Effective 

6 

6 

15 

11 

5 

0 

5 

5 

3 

1 

34 

23 

11.33 

6.76 

Ineffective 

3 

2 

3 

5 

1 

0 

0 

1 

2 

0 

9 

8 

3.00 

2.35 

inftrequgft  Events 

Effective 

0 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

1 

0.00 

0.29 

Ineffective 

5 

2 

2 

3 

0 

0 

0 

1 

2 

0 

9 

6 

3.00 

1.76 

Tafc&L 

Effective 

55 

64 

109 

65 

38 

10 

42 

39 

35 

13 

279 

191  93.00 

56.18 

Ineffective 

25 

20 

31 

38 

8 

12 

3 

12 

8 

5 

75 

87 

25.00 

25.59 
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discussed  in  Guerette  et  al.  <1987) .  It  is  possible  that  although 
the  instructors  find  the  same  level  of  errors  (indicated  here  by 
ineffective  behaviors) ,  perhaps  the  instructors  "look  for"  more 
positives  (indicated  here  by  effective  behaviors)  from  the  teams 
with  "good"  attitudes  that  perform  at  a  higher  level,  or  the 
differences  might  be  more  simply  explained  by  assuming  that  the 
higher  performers  outperform  the  less  effective  teams  because 
they  have  available  a  greater  number  of  effective  behaviors. 
Only  additional  research  can  clarify  the  inferences  to  be  made. 

Another  discrimination  effect  in  the  Critical  Team  Behaviors 
data  involved  the  information  flow  to  and  from  the  GLO  (team 
leader)  in  the  more  effective  and  less  effective  teams.  The  data 
relevant  to  this  finding  are  presented  in  Table  10.  This  table 
reports  by  phase,  and  separately  for  the  more  effective  and  less 
effective  teams,  the  frequencies  and  within-phase  percentages  of 
behaviors  for  which  the  GLO  was  an  initiator  or  recipient  of 
action. 


Table  10 

Percentages  of  Behaviors  by  Phase  for  the  GLO 
of  More  Effective  and  Less  Effective  Teams 


Phase 


GLO's 

Role 

Basics 

Pre- 

Midterm 

Midterm 

Post- 

Midterm 

Final 

Hsxe  Effective  Teams  (n 

-  3) 

Initiator 

22 

20 

14 

8 

6 

52.4% 

39.2% 

87.5% 

36.4% 

50.0% 

Recipient 

20 

31 

2 

14 

6 

47.6% 

60.8% 

12.5% 

63.6% 

50.0% 

Total 

42 

51 

16 

22 

12 

100.0% 

100.0% 

100.0% 

100.0% 

100.0% 

ftw  Effective  Teams  (n 

-  4) 

Initiator 

38 

33 

10 

20 

3 

70.4% 

68.8% 

62.5% 

66.7% 

50.0% 

Recipient 

16 

15 

6 

10 

3 

29.6% 

31.2% 

37.5% 

33.3% 

50.0% 

Total 

54 

48 

16 

30 

6 

100.0% 

100.0% 

100.0% 

100.0% 

100.0% 

Technical  Report  87-016 


These  data  indicate  that  there  was  a  differential  pattern  of 
information  flow  for  the  more  effective  and  less  effective  teams. 
In  the  less  effective  teams,  information  tended  to  flow  frpig  the 
GLO  to  the  other  team  members.  On  average,  a  GLO  in  these  teams 
initiated  about  68%  of  all  the  behaviors  in  which  he  was 
involved.  This  suggests  that  the  leaders  of  these  teams 
perceived  a  need  to  initiate  actions  to  compensate  for  the 
deficiencies  of  other  team  members,  and  that  they  adapted  a  more 
active,  directive  form  of  leadership.  In  the  more  effective 
teams,  the  leadership  pattern  seems  to  be  more  relaxed, 
particularly  in  the  three  " learning”  phases  of  training,  Basics, 
Pre-midterm,  and  Post-midterm,  on  average,  a  GLO  in  these  teams 
initiated  only  49%  of  the  behaviors  in  which  he  was  involved. 
This  suggests  that  these  team  members  more  often  "carry  their  own 
load"  during  performance  so  that  the  GLO  is  not  required  to 
initiate  most  of  the  action.  During  the  midterm,  however,  the 
effective  GLOs  seem  to  take  control  and  become  much  more  active 
participants  in  performance.  In  addition,  the  overall  pattern  of 
interaction  seems  to  be  disrupted  during  the  final  phase  when 
information  flows  equally  in  both  directions  for  both  groups  of 
teams.  These  important  differences  in  information  flow  might  have 
important  implications  concerning  the  effectiveness  of  different 
types  leadership. 


SELF-REPORT  QUESTIONNAIRE 

The  analysis  of  the  Trainee  Self-Report  Questionnaire  began 
with  105  data  points  for  each  of  the  117  individuals  in  the 
current  sample  (21  questionnaire  items  for  each  of  the  five 
phases  of  training) .  Estimates  of  the  within-phase  reliability 
of  the  questionnaire  were  obtained  by  computing  Coefficient  Alpha 
(Nunnally,  1978)  for  each  phase  of  training.  These  estimates  are 
given  in  Table  11.  Although  the  questionnaire  was  not  designed 
to  provide  samples  within  a  single  response  domain,  the  magnitude 
of  the  obtained  coefficients  indicates  that  the  instrument  did 
achieve  acceptable  levels  of  internal  consistency. 

Although  the  questionnaire  items  exhibit  reasonable  levels 
of  consistency  within  each  phase,  the  responses  to  the  items 
should  be  expected  to  undergo  changes  across  the  phases  of 
training.  As  training  proceeds,  job  skills  are  acquired,  team 
interaction  patterns  are  established,  lessons  learned  interact 
and  aggregate,  and  perceptions  are  transformed.  These 
experiences  should  produce  changes  in  one's  view  of  the  team  and 
his  understanding  of  the  questionnaire  items.  In  turn,  these 
changes  should  alter  the  pattern  of  intercorrelations  among  the 
items.  Thus,  the  21  items  cannot  be  expected  to  be  constant, 
discrete,  and  independent  measures.  Nevertheless,  a  "lower-limit" 
estimate  of  test-retest  reliability  was  obtained  by  computing  the 
correlation  between  the  responses  from  the  first  two 
administrations  of  the  questionnaire  (without  respect  to  phase) . 
This  procedure  yielded  a  reliability  coefficient  of  0.64  for  the 
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total  score  on  the  questionnaire.  While  this  estimate  must  be 
interpreted  with  considerable  caution,  it  also  suggests  that  the 
instrument  has  acceptable  psychometric  properties. 


Table  11 

Coefficients  Alpha  for  the  Self-Report 
Questionnaire  by  Phase 


Phase 

Coefficient 

Alpha 

1 

0.81 

2 

0.86 

3 

0.86 

4 

0.89 

5 

0.87 

factor  Analysis 

The  questionnaire  data  were  submitted  to  a  series  of 
exploratory  factor  analyses  designed  to  determine  the  extent  to 
which  the  factor  structure  of  questionnaire  responses  changed  as 
a  function  of  training.  It  was  reasoned  that  the  changing 
patterns  of  item  intercorrelations  over  time  would  be  reflected 
in  changes  in  the  number  and  the  contents  of  factors  that  emerged 
at  each  of  the  five  phases  of  training,  in  the  changes  in  item 
loadings  on  the  factors,  and  by  shifts  in  the  proportion  of 
variance  accounted  for  by  the  factors  that  emerged  at  different 
phases.  The  use  of  factor  analysis  was  also  deemed  appropriate 
because  it  would  help  to  (a)  reduce  the  number  of  variables  to  be 
manipulated  statistically  and  cognitively;  (b)  increase  the 
reliability  of  measurement  by  increasing  the  number  of  items 
incorporated  into  a  given  measure;  and  (c)  provide  a  meaningful 
explanation  of  the  evolution  of  teamwork  based  on  the 
interpretations  of  factor  structures.  The  classic  analog  for 
studies  of  this  sort  is  Fleishman  and  Hempel's  (1955)  work  on 
changes  in  psychomotor  skill  factor  structure  with  practice. 

The  factor  analyses  (principal  axis)  began  with  a  matrix  of 
intercorrelations  containing  all  of  the  items  from  all  of  the 
phases  (the  total  105  possible  data  points) .  This  overall 
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analysis  of  the  total  data  set  resulted  in  a  factor  structure 
with  five  factors,  each  of  which  contained  the  responses 
-obtained  during  one  of  the  five  phases  of  training  (thus,  a 
different  factor  represented  each  phase  of  training) .  This  was 
interpreted  to  mean  that  the  team  members  perceived  the  team 
differently  in  each  of  the  phases.  This  provided  evidence  that 
"something"  in  the  situation  changes  from  one  phase  to  another, 
that  this  change  is  detected  by  the  team  members,  and  that  this 
changing  perception  is  reflected  in  their  responses. 

This  overall  factor  analysis,  however,  provided  little 
insight  into  the  nature  of  the  changes  in  perceptions  during 
training.  Therefore,  a  series  of  factor  analyses  was  conducted 
in  which  the  data  from  adjacent  phases  were  combined  (Basics  with 
Premidterm,  Premidterm  with  Midterm,  and  so  forth)  and  a  separate 
factor  analysis  was  made  of  each  set  of  combined  data.  This 
approach  was  successful  in  providing  a  basis  for  interpreting  the 
chances  in  team  members'  perceptions  from  one  phase  to  another 
during  the  course  of  training. 

These  analyses  generated  three  primary  factors,  which  have 
been  designated  as  a  "teamwork"  factor,  a  "taskwork"  factor,  and 
a  "jelling"  factor.  (For  a  more  detailed  description  of  the 
factor  analysis  results,  see  Appendix  I) .  The  teamwork  and 
taskwork  factors  appear  to  correspond  to  the  two  activity  tracks 
of  the  TEAM  model  (see  Figure  2) .  As  anticipated  by  the  TEAM 
model,  these  factors,  which  are  separate  and  distinct  at  the 
beginning  of  training,  merge  into  a  single  factor  later  in 
training.  The  jelling  factor  emerges  during  the  later  phases 
(midterm  and  final) .  These  results  provide  empirical  support  for 
the  general  concepts  embodied  in  the  TEAM  model. 


profiles  &£  Perceptions 

After  the  teamwork,  taskwork  and  jelling  factors  had  been 
identified,  factor  scores  were  developed  to  represent  each  factor 
and  phase.  These  scores  were  based  on  factor  keys,  which  were 
designated  to  include  those  items  that  were  most  common  to  the 
respective  factors  across  phases.  (See  Appendix  I  for  a  fuller 
explanation  of  the  factor  scores  and  their  development) .  Thus, 
items  5,  9,  12,  18,  19  and  21  constituted  the  teamwork  factor 
key;  itema  4,  10,  11,  17  and  20  constituted  the  taskwork  factor 
key;  and  items  2,  3,  and  8  formed  the  jelling  factor  key.  (See 
Appendix  F  for  the  contents  of  self-report  items) .  The  factors 
were  used  to  obtain  scores  for  each  phase  by  summing  the  weights 
for  the  individual  items  that  make  up  each  factor  key.  Table 
12  presents  the  mean  factor  scores  by  phase  fi -  the  teams  in  the 
sample . 

These  factor  scores  can  be  compared  across  phases,  but  not 
with  each  other,  because  different  numbers  of  items  make  up  each 
of  the  keys.  An  examination  of  the  factor  scores  for  evolution 
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and  maturation  effects,  indicates  that  the  maximum  score  occurred 
in  the  midterm  phase  for  each  of  the  factors.  This  testing  phase 
is  the  first  and  most  significant  hurdle  in  the  successful 
completion  of  the  training,  and  it  would  appear  that  the  teams 
are  perceived  by  their  members  as  "rising  to  the  occasion"  with 
maximal  teamwork,  taskwork,  and  jelling  efforts.  It  should  be 
noted  that  virtually  all  teams  that  successfully  pass  the  midterm 
go  on  to  complete  the  training  course  successfully. 


Table  12 

Kean  Factor  Scores  for  Each  Phase  of  Training 


Training  Phase 

Factor  Identification 

Teamwork 

Taskwork 

Jelling 

Basics 

23.72 

17.52 

12.53 

Pre-Midterm 

24.65 

18.29 

12.54 

Midterm 

25.02 

18.69 

12.70 

Post-Midterm 

24.74 

18.39 

12.32 

Final 

24.45 

17.31 

12.48 

57 


Technical  Report  87-016 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK. 


58 


i 


Technical  Report  87-016 


DISCUSSION 


OVERVIEW  OF  FINDINGS 

This  section  will  focus  on  the  empirical  findings  that 
appear  to  contribute  most  to  the  understanding  of  how  Naval 
teams  develop  and  the  processes  by  which  teamwork  skills  evolve 
and  mature.  Whenever  appropriate,  in  order  to  add  insight 
concerning  the  nature  of  the  phenomena,  illustrations  will  also 
be  drawn  from  experiences  obtained  during  the  course  of 
developing  the  procedures  used  in  data  collection  and  analysis. 

In  the  next  and  final  section,  special  attention  will  be 
given  to  a  number  of  possible  applications  of  these  findings 
that  might  be  incorporated  into  the  design  of  training  systems 
and  instructional  techniques.  As  indicated  earlier,  the  ultimate 
goal  of  this  programmatic  research  effort  is  the  generation  of 
such  innovations  and  their  evaluation  against  the  criteria  of 
utility  and  training  effectiveness.  It  should  be  noted  that 
similar  studies  are  being  conducted  under  the  present  program  at 
Anti-Submarine  Warfare  (ASW)  and  Guided  Missile  Training  (GMS) 
installations.  These  subsequent  investigations  are  designed  to 
extend  the  range  of  situations  covered,  enable  comparisons  to  be 
made,  suggest  additional  potential  innovations,  and  provide 
inferences  as  to  the  generalizability  of  the  findings  and  the 
operational  applicability  of  recommended  innovations.  It  should 
also  be  noted  that  some  recent  changes  in  the  NGFS  curriculum 
have  moved  in  the  di<  actions  indicated  here. 

The  current  discussion  will  be  divided  into  three  parts. 
The  first  part  will  deal  with  findings  related  to  the  conceptual 
framework  around  which  the  NGFS  study  was  originally  organized. 
Highlights  of  the  findings  concerned  with  the  research's 
theoretical  concepts  may  be  summarized  as  follows: 

1.  Analyses  of  the  data  obtained  from  the  Critical  Team 
Behaviors  Form  and  the  Trainee  Self-Report  Questionnaire 
support  the  notion  that  teams  do  progress  through 
distinguishable  stages  of  development. 

2.  A  relatively  larger  number  of  effective  behaviors  were 
reported  for  the  more  effective  teams  than  for  less 
effective  teams.  Teams  were  approximately  equal  in  terms 
of  ineffective  behaviors. 

3.  Factor  analysis  of  the  Trainee  Self-Report  data 
identified  the  presence  of  a  teamwork  factor  and  a 
taskwork  factor.  This  suggests  that  these  components  of 
training  are  independent  and  that  sound  instructional 
design  should  specifically  address  each  one. 
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4.  Factor  analysis  of  these  data  indicated  that  teamwork 
was  more  salient  during  the  early  phases  of  training  and 
that  taskwork  gained  prominence  thereafter.  This 
suggests  that  teamwork  skills  such  as  cooperation  and 
coordination  should  be  stressed  during  the  early  stages 
of  training. 

5.  During  the  later  stages  of  development,  the  teamwork  and 
taskwork  factors  converged,  suggesting  that  optimal 
performance  requires  attention  to  both  team  and  task 
factors.  This  confirms  the  importance  of  maintaining  a 
focus  on  training  for  teamwork,  in  addition  to  a  task 
focus,  in  order  for  the  team  to  become  effective. 

6.  Greater  cohesiveness,  as  evidenced  by  the  "jelling" 
factor  was  noted  during  the  testing  phases.  It  is 
possible  that  the  imposition  of  an  external  demand  serves 
as  a  facilitator,  and  that  the  introduction  of 
measurable  goals  at  an  earlier  period  might  expedite 
the  development  of  cohesion  within  a  team. 

The  second  part  of  this  discussion  will  deal  with  the 
technical  and  methodological  aspects  of  the  data  collection, 
measurement,  and  analysis  techniques  employed  in  the  research. 
This  discussion  of  technological  and  methodological  matters  will 
focus  on  the  following  points 

1.  The  Critical  Team  Behaviors  Form,  which  provides  a  record 
of  observed  trainee  behaviors  as  perceived  by  the 
instructors,  was  found  to  be  sensitive  to  differences  in 
team  performance. 

The  third  part  of  this  discussion  will  focus  on  the 
implications  of  the  current  findings  for  improving  instructional 
methods.  Highlights  of  these  implications  include  the  following: 

1.  Individual  training  is  necessary,  but  not  sufficient  for 
effective  team  performance.  This  is  supported  by  results 
of  the  factor  analyses  which  suggest  that  it  would  be 
beneficial  to  focus  on  teamwork  skills  such  as 
cooperation  and  coordination  before  emphasizing  task- 
oriented  skills.  Thereafter,  integration  of  the  two 
types  of  skills  should  be  stressed. 

2.  Unexpected,  non-routine  events  should  be  incorporated 
into  training  programs  in  order  to  emphasize  the  need  for 
teamwork. 

3.  The  imposition  of  external  demands  along  with 
measures  of  goal  and  subgoal  achievement  might  expedite 
the  development  of  cohesion  within  teams. 
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4.  Behavioral  standards  identified  by  findings  from  the 
Critical  Team  Behaviors  Form  should  serve  as 
performance  benchmarks  for  goal-setting  and  for 
evaluating  the  performance  of  teams. 

5.  Comparisons  of  the  overall  performance  ratings  of  the 
team  with  the  individual  performance  ratings  enables  the 
identification  of  "critical  positions,"  in  which 
deficiencies  on  the  part  of  the  incumbents  may  not  be 
counteracted  by  additional  effort  by  other  team  members. 

6.  The  formal  leader  of  the  team,  the  GLO,  appears  to 

occupy  a  "critical  position,"  and  this  position  might 
benefit  from  supplemental  training  in  team  performance 
concepts  such  as  communication,  cooperation,  and 
coordination. 

7.  The  implications  from  this  research  should  be 
incorporated  into  a  training  program  to  enhance  the 
instructors'  understanding  of  the  developmental  process 
of  teams.  This  understanding  could  facilitate  the 
development  of  realistically  higher  expectations 
concerning  team  performance  at  different  stages  of 
development.  Furthermore,  training  on  the  use  of  the 
Critical  Team  Behaviors  Form  could  enhance  its  utility  as 
a  basis  for  diagnosis  and  providing  specific  feedback. 


CONCEPTUAL  FRAMEWORK  FOR  IMPROVING  TRAINING 

The  driving  force  behind  this  research  has  been  the  search 
for  ways  to  improve  team  training.  Such  efforts  necessitate  a 
conceptualization  of  the  developmental  processes  through  which 
teamwork  evolves.  Such  a  framework  provides  direction  and 
facilitates  the  systematic  exploration  and  evaluation  of 
alternative  interventions.  As  stated  at  the  outset,  a  failing  of 
earlier  team  training  research  has  been  the  lack  of  attention  to 
the  process  through  which  teamwork  skills  such  as  communication 
and  cooperation  are  learned  (Dyer,  1984;  Morgan  et  al.,  1986). 

Conceptual  efforts  related  to  the  present  research  have  been 
focused  on  two  major  objectives;  (1)  the  description  of  changes 
in  the  processes  evidenced  in  teams  undergoing  training;  and  (2) 
the  identification  of  interventions  that  might  expedite  and 
enhance  the  development  of  teams.  Therefore,  this  section  begins 
with  a  projection  of  the  current  findings  against  developmental 
models  depicting  the  change  processes.  These  findings  are  then 
discussed  in  terms  of  their  implications  for  improving  training. 

Efforts  to  understand  how  teams  evolve  under  the  present 
system  of  training  have  focused  on  identifying  the  changes  in 
team  behaviors  that  were  observed  and  recorded  by  the  instructors 
(using  the  Critical  Team  Behaviors  Form)  and  reported  by  the  team 
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members  (using  the  Trainee  Self-Report  Questionnaire) .  Both 
sources  of  data  reflect  changes  over  time  and  provided  support 
for  the  developmental  models  described  in  the  following 
paragraphs . 

Three  models  have  been  developed  during  the  course  of  this 
research  program.  The  Generalized  Model  of  Team  Evolution  and 
Maturation  (see  Figure  2)  and  the  stage  Model  of  Naval  Gunfire 
Support  Training  (see  Figure  4)  address  the  stages  through  which 
teams  progress  during  the  course  of  training.  The  Component 
Model  of  NGFS  Training  (Figure  3)  describes  how  instructors 
involve  themselves  as  a  "member"  of  the  team,  then  withdraw  from 
that  role  in  order  to  serve  as  an  outside  resource  and  transfer 
-  -  "ownership"  of  the  training  process  to  the  team  (see  Figure  3) . 
The  following  sections  will  relate  findings  from  NGFS  to  the 
Stage  Model  of  Naval  Gunfire  Support  Training.  Thereafter,  data 
supporting  the  Component  Model  of  NGFS  training  will  be 
discussed. 


Stage  Model  of  Naval  Gunfire  Support  Training 

A  major  question  addressed  by  this  investigation  has  been 
whether  or  not  teams  progress  through  identifiable  stages  of 
development  during  training.  The  results  of  this  investigation 
indicate  that  such  a  progression  can  be  identified.  This  holds 
implications  for  the  sequencing  of  training,  because  the  sequence 
of  training  activities  should  parallel  and  enhance  the 
development  of  teams.  For  instance,  it  would  be  unrealistic  to 
expect  that  team  members  would  be  able  to  work  together  optimally 
without  first  recognizing  and  understanding  the  nature  of  their 
interdependencies . 

Data  from  the  Trainee  Self-Report  Questionnaire  provided 
support  for  the  Stage  Model  of  Naval  Gunfire  Support  training. 
Factor  analyses  of  these  data  have  revealed  the  presence  of  both 
a  teamwork  factor  and  a  taskwork  factor.  The  former  reflects  an 
orientation  toward  the  development  of  team  interactions  and  the 
latter  indicates  an  orientation  toward  the  specific  requirements 
of  the  team's  technical  tasks.  As  training  progressed,  these  two 
factors  converged.  Both  the  presence  of  the  two  factors  and 
their  convergence  over  time  parallel  expectations  based  on  the 
TEAM  model  (Morgan  et  al.,  1986). 

Thus,  these  findings  confirm  the  existence  of  developmental 
stages  and  provide  information  concerning  the  nature  of  the 
changes  that  take  place  over  time.  Information  of  this  nature 
has  implications  for  the  training  of  instructors,  many  of  whom 
possess  strong  technical  backgrounds  and  general  instructional 
training,  but  have  had  little  formal  exposure  to  teamwork 
concepts.  Applications  of  these  findings  could  be  assimilated 
into  instructional  training  aids  to  increase  awareness  of  the 
importance  of  team  skills,  and  to  demonstrate  that  optimal 
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performance  requires  proficiency  in  both  teamwork  and  taskwork 
components.  One  possible  way  to  emphasize  the  crucial  nature  of 
teamwork  for  team  members  and  instructors  alike,  is  to 
incorporate  unexpected,  non-routine  events  into  the  training 
procedures . 

A  third  factor,  referred  to  as  the  "jelling"  factor,  emerged 
during  the  two  "testing"  phases  of  training.  This  factor 
reflects  group  cohesiveness,  and  it  is  possible  that  the 
introduction  of  an  external  demand,  the  tests,  promoted  its 
emergence.  Analysis  of  the  data  collected  at  ASW,  which  provides 
only  one  test  on  the  final  day  of  training  should  help  to  clarify 
whether  this  factor  is  influenced  by  the  imposition  of  an 
external  demand.  Evidence  supporting  the  latter  explanation 
would  argue  for  the  implementation  of  explicit  goals  and  subgoals 
at  earlier  stages  of  development,  perhaps  on  a  daily  or 
individual  exercise  basis.  The  inclusion  of  unexpected  events 
might  also  serve  to  expedite  the  development  of  cohesion  as  well 
as  promoting  the  convergence  of  teamwork  and  taskwork  factors. 


component  Modal  at  w<?fs  Training 

During  the  early  stages  of  training,  the  instructor  assumes 
a  key  role  in  the  team.  By  providing  direction  to  the  team 
members,  the  instructor  serves  as  a  role  model  for  the  team 
leader,  who  will  subsequently  be  expected  to  direct  the  team.  As 
training  progresses,  it  is  necessary  for  the  formal  leader  to 
assume  greater  responsibility  for  the  performance  of  the  team, 
while  the  instructor  withdraws  from  the  role  of  a  team  member  to 
assume  the  role  of  an  outside  observer  and  facilitator.  Initial 
interviews  with  the  instructors  suggested  that  this  transition 
tended  to  occur  around  the  time  of  the  midterm.  Data  collected 
with  the  Critical  Team  Behaviors  Form  (see  Table  8)  revealed  a 
sharp  drop  in  the  number  of  recorded  behaviors  involving  the 
instructor  after  the  pre-midterm  phase.  The  instructor 
maintained  a  low  level  of  direct  involvement  with  the  team  in  the 
last  three  phases. 


IMPROVING  INSTRUMENTS  AND  ANALYSES 

This  section  discusses  the  procedures  used  in  the  analysis 
of  the  changing  perceptions  of  the  team  members;  i.e.,  the 
introduction  of  the  factor  analysis  of  overlapping  sessions  of 
data.  Future  directions  for  the  analysis  of  the  present  data  are 
also  discussed.  The  Trainee  Self-Report  Questionnaire,  which 
provided  the  responses  for  the  overlapping  sessions  factor 
analysis,  is  also  described  and  suggestions  for  modifications  are 
offered.  Next,  attention  is  directed  to  the  Critical  Team 
Behaviors  Form,  which  served  as  a  record  of  instructor 
observations  of  the  behaviors  of  team  members.  The  potential 
utility  of  this  instrument  as  a  diagnostic  tool  for  instructors 
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is  discussed  along  with  suggestions  regarding  its  implementation. 
Finally,  the  use  of  the  rating  forms  to  assess  individual 
performance  in  relation  to  the  performance  of  the  team  as  a  whole 
is  discussed. 


Analytic  Techniques 

One  of  the  most  challenging  aspects  of  this  research  was  to 
develop  analytic  techniques  that  would  provide  a  means  of  tracing 
the  evolution  and  maturation  of  the  teams  in  training.  One  of 
the  most  important  results  of  these  efforts  was  the  development 
of  the  overlapping-sessions  factor  analyses  which  were  used  to 
analyze  the  responses  to  the  Trainee  Self-Report  Questionnaire. 
This  technique  appears  to  be  a  unique  application  of  factors 
analysis  procedures  (J.  L.  James,  personal  communication,  June  8, 
1987) .  No  reference  to  this  approach  has  been  discovered  in  the 
literature.  However,  the  current  application  does  meet  the 
proposed  guidelines  for  exploratory  factor  analysis  presented  by 
Ford,  MacCallum,  and  Tait  (1986). 

Beyond  the  analyses  of  the  data  recorded  in  this  report,  a 
continuing  effort  is  in  progress  to  analyze  the  current  data  in 
other  ways.  These  new  analyses  have  as  one  goal  a  method  for 
linking  the  results  of  the  different  data  sources  together  to 
portray  the  entire  process  in  a  multi-dimensional  space,  defined 
by  the  vector  components  of  the  different  solutions  of  the 
individual  data  sets.  As  appropriate,  the  results  of  these 
additional  analyses  will  be  presented  in  an  upcoming  technical 
report. 


pflta  SQUesfei-qn  and  Measurement  Method 

Two  types  of  information  were  collected  at  NGFS .  One  type 
of  data  reflected  the  observations  of  the  instructors,  while  the 
second  type  reflected  the  perceptions  of  the  team  members.  The 
first  type  of  information  was  collected  using  the  Critical  Team 
Behaviors  Form,  the  Individual  Performance  Summary,  and  the 
Overall  Performance  Form.  The  second  type  of  information  was 
obtained  from  the  Trainee  Self-Report  Questionnaire  and  the 
Individual  Performance  Summary  completed  by  the  Gunnery  Liaison 
Officer  (GLO) . 

Trainee  Self-Report  Questionnaire.  Training  that  achieves 
the  desired  results  requires  not  only  increases  in  the  number  of 
effective  behaviors,  but  also  changes  in  the  attitudes  and 
perceptions  of  the  trainees.  In  team  training,  it  is  necessary 
for  the  goals  of  the  trainees  to  be  transformed  from  an 
individual  focus  to  a  concern  with  the  periormance  of  the  team  as 
a  whole.  The  Trainee  Self-Report  Questionnaire  provides  a 
reliable  mechanism  for  tracking  this  change. 
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In  subsequent  versions  of  the  questionnaire  (e.g.,  for  use 
at  the  ASW  training  site) ,  sociometric  measures  involving  the 
"most"  and  "least"  valuable  player  on  the  team  have  been  deleted. 
These  items,  particularly  the  latter,  met  with  resistance  and 
derision  from  the  team  members.  Despite  assurances  of 
confidentiality,  most  of  the  respondents  felt  uncomfortable  with 
these  questions. 

Critical  Team  Behaviors  Form.  Attempts  to  improve  team 
training  require  the  ability  to  distinguish  between  more  and  less 
effective  teams.  Analysis  of  the  data  provided  by  The  Critical 
Team  Behaviors  Form  indicates  that  this  form  (or  variations 
thereof)  might  have  potential  as  a  diagnostic  and  debriefing 
tool.  However,  more  formalized  training  of  the  instructors  in 
the  utilization  of  this  form  would  certainly  enhance  its 
usefulness  as  such  a  tool.  Training  such  as  that  provided  to 
assessment  center  raters  could  be  used  to  enhance  the 
observation  and  coding  skills  of  the  instructors,  necessary 
elements  in  the  accurate  recording  of  performance  assets  and 
deficiencies.  Special  videotapes  of  team  performance  could  alBo 
be  employed  in  conjunction  with  "master"  Critical  Team  Behavior 
Forms,  which  would  indicate  which  of  the  critical  behaviors  were 
displayed  and  by  whom.  Portions  of  the  videotape  could  be 
replayed  to  highlight  observation  error.  These  approaches  are 
particularly  needed  in  order  to  deal  with  interpersonal  behaviors 
which  usually  are  not  as  easy  to  observe  as  the  technical  task 
components  of  behavior. 

For  practical  purposes,  it  would  be  desirable  to  reduce  the 
length  of  the  Critical  Team  Behaviors  Form.  The  demands  for  the 
broadest  possible  coverage  of  behaviors  in  the  first  round  of 
research  must  give  way  to  utilitarian  concerns  if  more 
operational  applications  of  this  technique  are  to  be  implemented. 
Whereas  the  Overall  Team  Performance  Form  and  the  Individual 
Performance  Summary  could  be  completed  in  three  to  five  minutes, 
the  Critical  Team  Behaviors  form  often  required  up  to  45  minutes. 
An  obvious  concern  is  that  the  quality  of  the  data  collected  may 
suffer  because  of  the  time  required  for  completion.  To  the 
extant  that  such  errors  exist,  it  is  likely  that  they  are  errors 
of  omission.  The  effects  of  such  errors  are  difficult  to 
predict.  Since  the  scoring  procedure  traditionally  used  by  the 
instructors  during  the  midterm  and  final  subtracts  points  from 
100  when  errors  are  made,  however,  it  xs  not  unreasonable  to 
assume  that  the  instructors  become  more  sensitive  to  ineffective 
behaviors  than  effective  ones.  Therefore,  a  disproportionate 
number  of  effective  behaviors  might  have  been  omitted.  It  is 
also  possible,  as  a  general  proposition,  that  incidents  of 
effective  performance  do  not  as  readily  elicit  reactions  from 
instructors  or  supervisors  simply  because  they  less  often  call 
for  action  by  the  observer  (they  are  perceived  as  less  critical 
incidents).  Commonly  this  is  rationalized  as  "If  it  isn't 
broken,  don't  fix  it."  Even  so,  it  is  the  number  of  effective 
behaviors  within  each  dimension  that  provides  the  distinction 
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between  more  and  less  effective  teams;  more  so  than  the  number  of 
ineffective  behaviors.  Both  effective  and  ineffective  behaviors, 
however,  do  show  potential  for  the  identification  of  benchmark 
critical  behaviors  for  purposes  of  discriminating  between  stages. 
A  related  innovation  would  be  a  device  that  makes  it  easier  to 
record  observations.  Hand-held  electronic  devices  exist  for 
making  entries  of  coded  information.  Some  permit  accumulated 
data  to  be  transferred  into  computer  files.  It  may  be  possible 
to  devise  such  an  aid  to  serve  this  purpose.  This  possibility  is 
being  explored. 

As  reported  earlier,  the  number  of  recorded  behaviors 
decreased  sharply  during  the  testing  phases  (i.e.,  the  midterm 
and  final) .  It  is  possible  that  the  team  members  displayed  fewer 
behaviors  during  these  phases.  However,  a  more  likely 
explanation  is  that  the  performance  criteria  on  which  the  teams 
were  being  tested  were  more  salient  to  the  instructors  and  that 
this  diverted  attention  away  from  the  observations  required  by 
the  Critical  Team  Behaviors  Form.  As  an  alternative  to 
collecting  data  during  the  testing  phases,  procedures  have  been 
established  at  subsequent  sites  (e.g.,  ASW)  to  allow  data 
collection  during  the  practice  mission  that  immediately  precedes 
testing.  (Two  or  three  such  missions  are  typically  included  for 
each  type  of  exercise.)  In  this  way  the  quality  of  the  data 
collected  could  be  improved. 


Other  Instruments.  An  important  consideration  in  team 
training  is  the  extent  to  which  extra  effort  on  the  part  of  some 
team  members  can  compensate  for  the  poorer  performance  of  other 
members.  The  combined  use  of  the  Instructor  Individual 
Performance  Summary  and  the  Overall  Team  Performance  Form 
provides  this  information.  Furthermore,  the  inclusion  of  the  GLO 
Individual  Performance  Summary  provides  a  means  for  assessing  the 
ability  of  the  formal  leader  to  evaluate  the  effectiveness  of 
individual  members  of  his  team.  This  ability  becomes  more 
essential  when  training  takes  place  aboard  ship,  under  the 
direction  of  ships'  officers  and  petty  officers. 

As  has  been  indicated,  the  ability  of  GLOs  to  judge  the 
performance  of  their  subordinates  appeared  to  be  inversely 
related  to  their  effectiveness  as  formal  leaders.  It  may  be  that 
the  cognitive  demands  placed  on  the  leader  in  effectively 
carrying  out  his  role  preclude  accurate  assessment.  This  being 
the  case,  it  may  be  productive  for  ships  engaging  in  shipboard 
training  to  provide  an  outside  observer  who  is  not  a  member  of 
the  formal  team  to  assume  the  role  of  an  evaluator. 

The  discussion  in  this  section  has  been  especially  sensitive 
to  situations  that  might  be  subject  to  improvement  by  one  means 
or  another.  This  may  have  had  untoward  consequences  for  the 
unwary  reader,  who  might  accentuate  the  negative.  It  would  be  a 
mistake  to  read  this  as  an  exercise  in  evaluating  NGFS  training. 
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Nearly  all  teams  undergoing  training  at  NGFS  complete  the 
training  and  meet  the  Navy's  standards  for  operational  readiness, 
and  all  of  the  current  data  came  from  ships  that  "passed"  the 
training  course.  The  aim  here  has  been  to  find  ways  to  make 
training  better.  Therefore,  the  discussion  has  focused  on  an 
effort  to  "look  for  trouble."  We  must  call  attention  to  the  fact 
that  the  NGFS  school  has  recently  initiated  a  number  of  changes 
in  the  organization  and  content  of  its  curriculum  that  move  in 
the  directions  indicated  here.  The  next  phase  of  this  research, 
will  accentuate  the  positive  as  efforts  are  made  to  design  and 
evaluate  specific  ways  to  raise  the  level  of  effectiveness  of 
training  for  teams  in  schools  and  in  ships,  some  potential  ways 
to  accomplish  this  objective  are  discussed  in  the  next  section. 


IMPROVING  METHODS  OF  INSTRUCTION 

It  is  difficult  to  suggest  improvements  in  the  training 
process  without  considering  the  role  of  the  instructor,  who 
provides  the  training.  For  this  reason,  the  following  discussion 
is  concerned  with  ways  to  help  the  instruc  or  to  improve  the 
quality  of  team  training. 

To  begin  with,  it  is  obvious  that  instructional  programs  for 
team  training  should  be  able  to  adapt  to  differences  in  the  needs 
of  the  trainees,  and  to  changes  that  take  place  in  teams  over 
time.  Further  refinement  and  adaptations  of  information  like 
that  of  the  Critical  Team  Behaviors  Form  could  provide  the  basis 
for  this  adaptability.  That  is,  instructors  could  be  trained  to 
use  this  or  similar  instruments  to  identify  deficits  in  teamwork 
skills,  diagnose  potential  causes  of  poor  performance,  and 
develop  standards  of  teamwork  for  each  stage  of  training.  Such 
information  could  be  used  by  instructors  to  modify  training 
regimens  as  necessary  to  address  identified  teamwork  problems. 

Although  the  central  interest  here  is  team  training,  the 
contribution  that  individual  competencies  make  to  the  success  of 
training  cannot  be  ignored.  Certain  minimal  levels  appear  to  be 
prerequisite  for  effective  team  training,  (Dyer,  1984;  Hall  and 
Rizzo,  1975;  Johnston,  1966).  For  example,  in  the  case  of 
NGFS  teams,  the  performance  of  the  whole  team  may  be  most  largely 
determined  by  the  competencies  of  the  Gunnery  Liaison  officer 
(GLO) .  Current  results  indicate,  however,  that  under  certain 
circumstances  other  team  members  can  compensate  for  weaknesses  in 
the  GLO.  Thus,  individual  competence  on  the  part  of  team  members 
is  a  necessary,  but  not  sufficient  condition  of  effective  team 
performance. 

Finally,  it  should  be  noted  that  the  training  of  the 
instructors  represents  the  culmination  of  all  other  efforts  to 
build  better  training  systems.  All  that  is  learned  about  the 
effectiveness  of  any  type  of  intervention  must  eventually  be 
incorporated  into  the  training  of  those  people  whose  ultimate 
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responsibility  it  will  be  to  teach  teamwork  skills.  such 
training  should  prepare  the  instructor  to  recognize,  reinforce, 
and  remedy  deficiencies  in  teamwork  skills. 


Sequencing  g£  Training 

When  planning  a  program  of  instruction  for  team  training,  it 
is  necessary  to  consider  the  team's  overall  readiness  for 
specific  types  of  training.  Factor  analyses  of  the  responses  to 
the  Trainee  Self-Report  Questionnaire  have  indicated  a 
differential  sensitivity  to  the  teamwork  and  taskwork  aspects  of 
training  across  phases.  During  the  initial  phase  of  training, 
the  teamwork  factor  accounted  for  the  largest  percentage  of  the 
variance.  In  the  following  phase,  the  taskwork  factor  accounted 
for  a  greater  proportion  of  the  variance.  This  shift  in  variance 
accounted  for  during  the  first  half  of  training  suggests  a  shift 
in  the  salience  of  each  factor  for  the  trainees.  After  the 
individual  members  were  brought  together  as  a  team,  the  nature  of 
their  interdependencies  became  salient,  and  this  cognizance  then 
allowed  the  team  to  focus  more  fully  on  the  activities  related  to 
technical  task  performance.  For  application  to  NGFS  training,  it 
is  suggested  that  teamwork  skills  such  as  cooperation  and 
coordination  should  be  emphasized  early  in  training  (beginning 
with  the  Basics  phase)  when  the  team  members  are  more  sensitive 
to  the  interdependencies  of  their  individual  responsibilities. 
Later,  when  the  focus  shifts  to  taskwork  factors,  emphasis  should 
be  placed  on  developing  the  technical  skills  which  promote  task 
accomplishment.  Further  testing  of  this  "critical  period" 
hypothesis  (e.g.,  Gersick,  1985)  of  development  is  still  needed. 


Expediting  Evaluation 

Constructs  such  as  cooperation  and  cohesiveness  are 
uniformly  linked  to  effective  team  performance.  Targeted  efforts 
to  expedite  their  emergence  could,  therefore,  be  useful  in 
improving  the  quality  of  team  performance  in  training.  The 
factor  analyses  of  the  Trainee  Self-Report  Questionnaire 
responses  suggests  that  this  might  be  possible.  The  emergence  of 
the  "jelling"  factor  (see  Table  11)  during  the  midterm  and  final 
phases  of  training  may  very  well  reflect  a  feeling  of  group 
solidarity  or  cohesiveness  that  results  when  the  team  members 
see  the  "pieces — taskwork  and  teamwork — fall  into  place"  as  they 
respond  to  the  requirements  of  collective  effort  and  achievements 
called  forth  by  the  "test  demands."  The  emergence  of  this  factor 
at  these  points  in  time  is  consistent  with  the  work  of  Gersick 
(1985) .  She  suggested  that  problem  solving  groups  tend  to  pull 
together  at  such  times  in  order  to  satisfy  external  demands.  It 
is  possible  that  the  daily  administration  of  tests,  or  other 
direct  indications  of  goal  and  subgoal  requirements  and 
achievements  (a  proposition  presently  being  operationalized  by 
NGFS  management),  might  expodite  the  developmental  process. 
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This  night  also  explain  the  emergence  of  the  taskwork  factor 
as  the  team  approached  the  midterm.  In  her  research  on  problem¬ 
solving  groups,  Gersick  found  that  groups  undergo  a 
transformation  halfway  through  their  lifecycle.  At  that  time, 
groups  re-evaluate  their  progress  and  refocus  their  energy  toward 
completion  of  the  task.  Initially,  it  was  assumed  that  this  type 
of  reformulation  would  occur  at  NGFS  during  the  midterm. 
Subsequent  evidence  concerning  the  importance  of  satisfactory 
performance  on  the  midterm,  and  the  recognition  that  the  bulk  of 
the  basic  training  material  is  presented  during  the  first  two 
phases  of  training,  challenged  this  assumption.  Rather,  it  might 
be  that  the  team  views  the  midterm  (rather  than  the  final) as  the 
"real"  test  .  That  being  the  case,  the  reforming  stage  might  be 
expected  to  occur  during  the  pre-midterm  phase.  It  is  during  the 
transition  from  the  pre-midterm  phase  to  the  midterm  phase  that 
the  taskwork  factor  emerges,  a  natural  consequence  of  the 
reformulation  and  a  precursor  to  the  development  of  the  "jelling" 
factor.  Data  to  be  collected  later  at  the  ASW  training  site  may 
help  to  clarify  whether  the  emergence  of  the  "jelling"  factor  is 
the  result  of  natural  evolution  and/or  the  product  of  an  external 
demand.  At  ASW,  the  only  formal  test  of  team  performance  occurs 
on  the  final  day  of  training. 


Developing  Standards 

The  identification  of  stages  in  the  process  of  team 
motivations,  and  interventions  which  may  speed  this  process, 
paves  the  way  for  the  development  of  teamwork  norms  for  teams  at 
various  performance  levels.  Greater  attention  needs  to  be 
directed  toward  the  identification  of  specific  changes  in 
behavior  which  could  serve  as  benchmarks  in  each  stage  of  team 
development.  For  instance,  such  changes  were  identified  in  the 
behaviors  recorded  in  the  Communications  dimension  of  the 
Critical  Team  Behaviors  Form.  During  the  initial  stages  of 
training,  team  members  are  frequently  unsure  about  how  they 
should  interact.  During  the  first  day  of  training,  one  of  the 
most  frequently  recorded  ineffective  behaviors  involves  team 
members  lowering  their  voices  and  mumbling  when  communicating 
information.  After  the  second  day  of  training,  this  behavior  is 
almost  nonexistent.  This  reflects  a  basic  problem  for  developing 
teams,  who  often  do  not  communicate  needed  information  and  lower 
their  voices  when  they  do  communicate  so  that  others  will  not 
detect  their  errors.  Thus,  during  tne  first  phase  of  training, 
trainees  should  be  instructed  that  certain  kinds  of  errors  are 
expected  to  occur  at  the  beginning  of  training,  and  that  the 
instructor  will  be  sensitive  to  their  feelings  of  confusion  and 
uncertainty. 

By  the  second  day  of  training,  team  members  appear  to  become 
more  comfortable  with  the  communication  process.  However,  as  the 
number  of  communications  increases— the  desirable  outcome — so 
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does  the  number  of  errors;  the  opportunity  for  making  errors  goes 
up  (see  Tables  7  &  9).  The  most  frequently  recorded 
communication  errors  at  this  stage  and  after  involve  the  use  of 
improper  terminology  and  the  failure  to  transmit  information  in 
the  proper  order.  By  the  time  of  the  midterm,  which  usually 
occurs  on  the  third  day  of  training,  the  number  of  communication 
errors  diminishes,  falling  below  the  level  reported  the  first 
day.  Greater  awareness  by  the  instructor  of  what  is  to  be 
expected  (e.g. ,  errors)  at  different  points  in  training  should 
enable  him  to  adapt  his  instructional  approach  more  appropriately 
to  the  changing  needs  of  the  teams. 

The  development  of  effective  communication  behaviors  appears 
to  be  particularly  crucial  for  team  success.  Throughout  the 
course  of  training,  between  44%  and  51%  of  the  recorded 
ineffective  behaviors  reflect  problems  in  communication.  This, 
no  doubt,  reflects  the  importance  attached  by  instructors  to 
effective  communication.  Many  instructors  reported  that  this  is 
the  most  important  dimension. 


Increasing  Individual  gprepAtenoe 

With  respect  to  team  performance,  it  has  often  been  stated 
that  "the  whole  is  greater  than  the  sum  of  its  parts."  However, 
the  individuals  comprising  the  team  must  come  to  training  with 
some  threshold  level  of  individual  competence  in  order  for  team 
training  to  be  efficient.  For  example,  plotters  must  know  how  to 
plot  both  the  movement  of  the  ship  and  the  location  of  the 
target,  or  what  other  team  members  do  cannot  have  much  effect 
upon  accomplishment  of  the  mission. 

Theoretically,  team  members,  should  have  reached  soma 
minimal  level  of  individual  competence  before  they  report  for 
training.  From  time  to  time,  however,  trainees  do  report  for 
team  training  without  adequate  individual  skills.  Indeed,  a  crew 
with  all  members  ready  for  team  training  is  a  rarity.  Interviews 
with  the  instructors  indicated  that  two  courses  of  action  are 
available:  (1)  the  instructor  can  take  time  to  teach  the 
individual  skills,  thereby  reducing  the  time  available  for  team 
training  (a  common  practice),  or  (2)  the  team  member  may  be  sent 
back  to  the  ship  and  a  replacement  may  be  requested  (a  rare 
event) .  The  viability  of  the  two  alternatives  is  likely  to 
depend  on  the  extent  of  the  deficiency.  This  recognition  and 
practice  is  in  agreement  with  the  assertions  of  Dyer  (1984),  Hall 
and  Rizzo  (1975),  and  Johnston  (1966)  regarding  the  necessity  for 
adequate  individual  competence  and,  as  in  this  investigation, 
that  of  teamwork  concepts.  Indeed,  both  are  needed  if  team 
training  is  to  be  successful. 

Even  if  all  team  members  possess  at  least  minimal 
competency,  some  trainees  wi!l  be  more  skilled  than  others.  An 
important  consideration  in  team  training  is  the  extent  to  which 


70 


Technical  Report  87-016 


trainees  can  compensate  for  one  another's  weaknesses  or  build  on 
one  another's  strengths.  Lower  levels  of  competency  might  be 
tolerated  in  the  noncritical  positions  if  the  remaining  members 
are  sufliciently  skilled  to  compensate  for  the  deficiency.  In 
the  event  that  the  remaining  members  lack  sufficient  skill  to 
permit  such  compensation ,  or  if  the  trainee  in  question  occupies 
a  "critical"  role,  the  instructor  may  be  forced  to  seek  a 
replacement. 


Preparing  Tsana  Ls&Sfirfi 

Individual  competence  appears  to  be  particularly  important 
in  the  case  of  the  formal  leader  of  a  team,  in  this  instance  the 
Gunnery  Liaison  Officer  (GLO) .  Data  reported  in  Figure  7  and 
Table  10  of  the  Results  section  suggests  a  vital  relationship 
between  the  individual  performance  of  the  GLO  and  the  performance 
of  the  team  as  a  whole.  In  the  more  effective  teams,  the 
instructors'  ratings  of  the  GLO  increased  through  the  midterm  and 
remained  relatively  high  thereafter  (Figure  7).  It  is 
interesting  to  note  that  their  highest  rating  occurred  during  the 
midterm  when  they  were  initiating  the  greatest  proportion  of 
their  behaviors  (Table  10) .  By  contrast,  the  GLOs  of  less 
effective  teams  received  consistently  low  ratings  even  though 
they  initiated  most  of  their  behaviors.  These  results  suggest 
that  special  attention  needs  to  be  directed  toward  the  training 
of  the  GLO.  It  might  be  advantageous  to  supplement  the  GLO's 
training  with  specific  instruction  in  team  performance  concepts 
such  as  effective  communication,  coordination,  and  cooperation. 
Such  instruction  should  deal  specifically  with  the  proper  (e.g., 
"diplomatic"  and  effective)  timing  and  delivery  of  initiated 
behaviors.  This  would  help  to  increase  the  likelihood  that  the 
formal  leader  will  have  learned  the  skills  required  to  assume  a 
full  leadership  role  as  the  team  approached  the  midterm,  the 
point  when  the  instructor  is  preparing  to  reduce  his  involvement 
in  the  team  and  assume  the  role  of  an  observer  and  resource 
person. 

Vaill  (1978)  has  suggested  that  the  importance  of  a 
leader '8  technical  expertise  has  tended  to  be  undervalued. 
Likewise,  Yukl  (1981)  has  identified  a  number  of  contingencies 
that  require  problem-solving  skills  of  the  leader.  It  seems 
reasonable  to  assume  that  problem-solving  skills  in  a  given  area 
are  preceded  by  technical  competence.  One  of  the  contingencies 
described  by  Yukl  occurs  when  the  group  operates  in  a  hostile 
environment  which  on  a  periodic  basis  endangers  its  existence. 
The  application  to  a  military  context  is  obvious. 


A  crucial  component  of  effective  team  training  is  diagnosis 
of  the  teams  deficiencies.  Once  a  diagnosis  is  made,  the 
training  program  can  be  modified  to  provide  remedies  in  areas  of 
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weakness.  As  indicated  earlier,  the  Critical  Team  Behaviors  Form 
can  be  of  considerable  assistance  in  this  regard.  As  shown  in 
Table  9  of  the  Results  section,  this  instrument  is  sensitive  to 
differences  in  the  performance  of  more  effective  and  less 
effective  teams.  Specifically,  a  greater  number  of  effective 
behaviors  were  recorded  for  teams  who  subsequently  performed  well 
on  the  final  exam  and  the  live  firing  than  for  those  teams  whose 
performance  was  less  effective;  no  meaningful  differences  were 
detected  for  ineffective  behaviors.  It  should  be  noted  that 
all  of  the  teams  included  in  the  study  successfully  completed  the 
final,  indicating  that  the  instrument  is  sensitive  to  differences 
despite  a  restriction  in  range. 

Diagnosis  using  this  instrument  is  facilitated  by  the 
categorization  of  behaviors  into  the  seven  dimensions  of  teamwork 
skills:  communication,  cooperation,  team  spirit  and  morale, 
giving  suggestions  or  criticisms,  accepting  suggestions  or 
criticisms,  coordination,  and  adaptability.  Thus,  patterns  of 
effective  and  ineffective  behaviors  in  specific  skill  areas  could 
be  detected,  and  instructors  could  made  aware  of  which  skills  and 
knowledges  require  increased  attention.  Furthermore,  the 
repeated  use  of  this  instrument  during  the  course  of  training 
could  provide  feedback  to  the  instructor  regarding  success  in 
avoiding  problems  or  correcting  deficiencies.  This  would  provide 
further  alerting  as  to  the  need  to  modify  his  instructional 
approach  based  on  the  progress  of  the  team  (or  lack  thereof) . 

The  behavioral  items  comprising  the  Critical  Team  Behaviors 
Form  were  provided  by  the  instructors,  using  their  terminology. 
This  contributes  to  the  meaningfulness  and  relevance  of  the  items 
for  the  instructors,  while  also  serving  to  increase  the 
instructors'  sensitivity  to  specific  behaviors  which  could  lead 
to  enhanced  or  reduced  performance.  It  is  also  worth  noting  that 
with  practice  the  form,  despite  its  length,  becomes  easier  to 
use.  In  addition,  there  is  the  possibility  of  using  a  trouble 
shooting  approach,  incorporated  in  a  handy  guide  and/or  computer, 
to  which  entry  is  made  by  problem  or  symptom  to  uncover  relevant 
underlying  behaviors  and  action  alternatives. 


Training  £&&  Instructors 

The  generalization  of  the  original  model  of  team  evolution 
and  maturation  to  the  NGFS  site  provides  the  basic  framework  for 
the  development  of  a  program  of  instruction  for  instructors. 
This  framework,  in  conjunction  with  evidence  for  a  shift  in  the 
salience  of  teamwork  and  taskwork  factors,  provides  the  basis  for 
the  development  of  norms  describing  the  typical  development  of 
teamwork  during  the  course  of  training.  Once  behavioral  norms 
are  established  for  various  levels  of  team  performance,  the 
information  can  be  presented  to  instructors  as  part  of  their  own 
training.  Instructors  need  to  know  the  types  of  behaviors,  both 
effective  and  ineffective,  that  can  be  expected  to  occur  at  each 
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stage  of  development.  Critical  behaviors  can  serve  as  benchmarks 
for  tracking  the  progress  of  each  team  undergoing  training. 
Furthermore,  the  effects  of  interventions,  such  as  the  imposition 
of  external  demands,  should  be  presented  as  tools  to  be  used  to 
expedite  team  development.  Videotapes  of  typical  performance 
could  be  used  to  provide  realism  absent  in  the  spoken  or  printed 
word.  Furthermore,  videotapes  developed  for  training  instructors 
could  also  be  shown  to  team  members,  thereby  providing  goals 
which  are  understandable  and  meaningful  to  the  team. 

The  existence  of  "critical"  versus  "noncritical"  team 
positions  should  also  be  discussed  and  training  should  be 
provided  to  assist  the  instructors  in  identifying  prerequisite 
threshold  levels  of  individual  performance.  Based  on  the 
establishment  of  minimal  criteria,  decision  rules  should  be 
developed  to  provide  guidance  in  handling  the  problem  of 
deficiencies  in  the  individual  skills  of  members.  Objective 
measures  of  individual  performance,  which  could  identify 
deficiencies  early  on,  would  provide  the  instructors  with  greater 
versatility  in  remedying  weaknesses. 
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RECOMMENDATIONS 

Because  this  research  was  conducted  within  the  context  of 
school  house  training,  most  of  the  discussion  has  addressed  the 
school  situations.  However,  it  must  be  noted  that  a  major  share 
of  training  takes  place  aboard  the  Navy's  ships  and  stations 
under  operational  command.  It  should  also  be  recognized  that 
embedded  training  capability  is  being  designed  into  new 
equipment,  and  this  capability  is  becoming  an  ever  enlarging 
component  of  the  Navy's  total  training  system.  Thus,  the  reader 
should  be  reminded  that  applications  of  the  current  findings, 
such  as  the  interventions  suggested  below,  will  inevitably  and 
increasingly  extend  beyond  school  sites  and  become  a  part  of  the 
everyday  concerns  of  operational  units. 


OVERVIEW 

This  section  focuses  on  the  development  of  interventions  to 
improve  team  training.  Its  major  purpose  is  to  outline  the  range 
of  potential  interventions  that  might  be  considered  for 
evaluation  during  the  next  phase  of  this  research  project.  The 
discussion  will  not  be  exhaustive,  but  it  will  present  a 
representative  sample  of  the  changes  that  are  suggested  by  the 
current  set  of  data.  For  purposes  of  this  discussion,  training 
systems  will  be  defined  as  consisting  of  three  major  components: 
1)  instructional  materials--such  as  the  curriculum  training 
objectives;  2)  instructional  technology— such  as  the  NGFS 
trainer;  3)  instructors — people  assigned  to  carry  out  the 
training.  Because  a  given  intervention  is  expected  to  impact  one 
of  these  components  primarily,  the  following  discussion  of 
interventions  will  be  divided  into  three  sections  corresponding 
to  these  three  categories  of  training  system  components.  However, 
this  categorization  is  somewhat  arbitrary  because,  given  the 
interdependent  nature  of  team  training  systems,  almost  all 
interventions  must  ultimately  be  expected  to  affect  the  entire 
training  system.  The  interventions  to  be  discussed  in  the 
following  sections  are  summarized  below.  Each  type  of 
intervention  will  be  considered  separately,  and  specific  examples 
of  each  will  be  provided. 


Instructional  Materials 

*  Modify  the  curriculum  to  include  additional  instruction 
concerning  team  interaction  processes. 

*  Initiate  new  testing  and/or  goal  setting  procedures 
to  increase  team  attention  to  taskwork  and  teamwork 
ob j ectives . 

*  Integrate  teamwork  concepts  with  simulator  exercises  to 
enhance  the  practice  of  teamwork  processes. 
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Instructional  ZsghnPlogv 

*  Install  audio  and  video  recording  systems  to  assist  the 
instructor  during  diagnosis,  evaluation,  and  feedback. 

*  Modify  and  install  the  IDAFT  (Instructional  Development 
of  Automated  Feedback  Training)  system  in  order  to 
provide  feedback  that  is  more  timely  and  relevant  to 
teamwork  skills. 

*  Develop  other  performance  measurement  tools  (e.g.,  coding 
devices)  to  record  behaviors  in  real  time  during 
training. 


Instructors 

*  Provide  additional  specialized  pipeline  training  in  team 
oriented  concepts  for  instructors  who  conduct  team 
training. 

*  Train  instructors  in  the  use  of  the  TEAM  measurement 
instruments  with  refinements  and  modifications  to  fit 
specific  missions  that  have  been  developed  for  use  in 
diagnosis,  feedback,  and  evaluation. 


INSTRUCTIONAL  MATERIALS 

Several  curriculum  changes  might  be  considered  as 
enhancements  to  current  team  training  systems.  The  general 
approaches  suggested  here  are  not  new.  However,  because  the  TEAM 
data  collection  instruments  and  analytic  procedures  provide  new 
methods  for  monitoring  the  the  development  of  teams  during 
training,  these  changes  warrant  new  application  to  team  training. 
Wexley  and  Baldwin  (1986)  have  examined  three  post-training 
strategies  which  are  designed  to  facilitate  positive  transfer  of 
training  to  the  job.  Specifically,  they  evaluated  strategies 
involving  goal-setting,  participative  goal-setting  (cf.,  Anderson 
&  Wexley,  1983),  and  relapse  prevention  self -management  (cf., 
Marx,  1986).  Wexley  and  Baldwin’s  methodology  could  be  applied 
in  the  current  TEAM  project  to  test  the  effects  of  these  types  of 
strategies  on  team  training. 

Another  potential  curriculum  intervention  would  require  the 
incorporation  of  principles  concerning  the  evolution  and 
maturation  of  teams  into  the  introductory  portions  of  training. 
Given  the  current  evidence  for  the  impact  of  teamwork  factors 
during  training,  the  highlighting  of  teamwork  principles  might  be 
expected  to  increase  the  salience  of  teamwork  processes  and 
quicken  the  evolution  and  maturation  of  teams.  Most  training 
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focuses  on  the  taskwork  (or  technical)  aspects  of  training,  with 
little  or  no  emphasis  on  the  variables  that  contribute  to 
teamwork.  Observations  by  experimenters  in  the  current 
investigation  have  indicated  that  team  building  (teamwork 
development)  usually  occurred  on  an  &£  hoc  basis  during  training. 
Formalizing  the  time  and  methods  for  team  building,  especially 
during  training  periods  when  teamwork  has  the  highest  salience 
for  team  members  (see  the  Discussion  section)  could  have  a 
profound  impact  on  training  effectiveness. 


Testing 

Given  (a)  the  existence  of  the  "jelling"  factor  during  the 
testing  phases  of  NGFS  training,  and  (b)  reports  from  the 
instructors  that  teams  seemed  to  become  more  unified  at  the 
midterm  phase,  it  is  possible  that  altering  the  testing 
procedures  might  have  a  positive  impact  on  training.  This  is 
consistent  with  Gersick's  (1985)  observation  that  groups  seem  to 
redirect  their  energy  as  the  nature  of  external  demands  become 
more  salient  to  them.  The  establishment  of  a  testing  period 
earlier  in  training  might  provide  a  catalyst  for  the  team  to  jell 
earlier,  thus  hastening  the  merger  of  the  teamwork  and  taskwork 
factors. 

A  second  testing  intervention  could  consist  of  the 
implementation  of  a  pretest  of  individual  skills  and  knowledges 
to  assist  instructors  in  the  initial  evaluation  of  a  team's 
capability.  Such  a  test  could  also  include  measures  of  team 
performance  levels  that  would  be  useful  to  the  instructors  in 
choosing  their  instructional  strategy  (see  Guerette,  et  al., 
1987) .  NGFS  instructors  indicated  that  the  implementation  of  a 
pretest  is  highly  desirable,  and  not  readily  available  in  the 
current  system. 


SlnmlfltQr  ExercX-g-ga 

As  noted  in  the  Discussion  section,  the  presentation  of 
novel  situations  during  training  can  have  a  positive  impact  on 
team  shaping  processes.  Such  novel  events  could  be  added  to  the 
existing  sets  of  simulator  exercises  to  advance  the  team 
maturation  process. 


INSTRUCTIONAL  TECHNOLOGY 
Audio  And  video  EqvlPfflgnt 

Experimentation  with  the  use  of  video  cameras  in  the 
current  research  has  added  some  heuristic  value  relevant  to 
potential  interventions.  This  experience  suggested  that  two 
intervention  strategies  are  possible.  First,  it  was  noted  by  the 
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instructors  that  the  videotapes  provided  information  concerning 
behaviors  that  were  missed  in  real-time  training.  Thus, 
videotape  review  of  training  sessions  might  improve  the 
instructor's  preparation  for  team  performance  feedback.  Second, 
the  development  of  video  samples  of  desired  behaviors  for  given 
exercises  could  be  prepared  (perhaps  with  instructors  in  each 
team  position)  in  order  to  provide  behaviors  which  could  be 
modeled  by  the  teams  in  training  in  a  feedforward  type  paradigm 
(as  well  as  to  illustrate  common  defects  to  be  avoided). 
Likewise,  the  ability  to  record  the  communications  that  occur 
during  training  sessions  could  provide  vital  information  for 
instructors  in  preparing  performance  feedback. 


IDAFT 


The  Instructional  Development  of  Automated  Feedback  Training 
(IDAFT)  system  originally  developed  (Andrews  &  Uliano,  in  press) 
for  the  Fleet  ASW  training  setting  is  proposed  as  a  source  of 
feedback  information  for  instructors.  The  existing  system 
focuses  on  individual  performance  variables,  and  it  would  have  to 
be  modified  to  include  team  performance  parameters.  Such  a 
modification  seems  feasible.  IDAFT  could  provide  an  additional 
source  of  performance  feedback  information,  especially  if  the 
modifications  included  both  taskwork  and  teamwork  elements. 


Coding  Pfiyisaa 

As  discussed  earlier,  the  fact  that  the  Critical  Team 
Behaviors  Form  cannot  be  completed  in  real  time,  and  the  length 
of  time  required  to  complete  the  form,  both  need  to  be  considered 
for  change  in  future  research  and  implementation  in  training 
settings.  An  intervention  that  might  solve  these  problems  is  the 
use  of  a  hand-held  coding  device  (similar  to  inventory  control 
devices  used  in  supermarkets) .  Using  such  a  device.  Instructors 
should  be  better  able  to  record  behaviors  in  real  time.  These 
could  then  be  complied  and  printed  out  at  the  end  of  a  training 
session. 


INSTRUCTORS 
pipeline  Training 

NGFS  instructors  currently  receive  three  weeks  of  training  at 
the  Instructor  Training  School.  As  a  part  of  this  training,  it 
would  seem  worthwhile  to  provide  instructors  with  additional 
training  on  the  general  team  development  process  principles  being 
uncovered  by  the  current  TEAM  research  effort.  This  additional 
training  could  include  specific  training  on  the  curriculum  and 
hardware  interventions  that  might  be  implemented  as  enhancements 
to  the  current  system.  Also,  an  emphasis  on  the  factors 


78 


Technical  Report  87-016 


identified  in  the  TEAM  model  could  increase  the  awareness  of 
teamwork  oriented  skill  requirements  as  well  as  technical  task 
skill  requirements,  and  enhance  the  understanding  of  their  joint 
contribution  to  successful  team  training. 


CONCLUSIONS 

As  was  stated  at  the  outset,  the  interventions 
highlighted  above  are  neither  all  inclusive  or  highly  detailed. 
They  are  presented  to  indicate  how  the  results  of  the  NGPS 
portion  of  the  TEAM  project  can  be  translated  directly  into 
interventions  which  could  substantially  improve  team  performance, 
both  in  the  short  term  and  ultimately  in  the  long-term.  The  next 
phase  of  the  TEAM  project  will  consist  of  (1)  the  selection  of 
testable  interventions  based  upon  a  comprehensive  review  of 
potential  interventions  in  association  with  experts  within  and 
outside  of  the  Navy,  and  (b)  measurement  of  the  effectiveness  of 
the  interventions  that  are  selected  for  implementation.  Based  on 
the  findings  of  that  research,  more  definitive  recommendations 
will  be  made  concerning  the  relative  efficacy  of  various  team 
training  interventions  in  current  and  future  training  systems. 
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APPENDIX  A 

Criteria  for  the  Selection  of  a 
Data  Collection  Site 


Primary  characteristics 

Member  Communication 

*  Extensive  (verbal  and  nonverbal) 

*  Observable  or  trackable  as  much  as  possible 
for  as  many  members  as  possible 

Member  Interaction 

*  Significant  periods  of  performance  time  (more 
than  5  minutes) 

*  At  least  2  members  interacting  at  any  one 
time  throughout  task 

Member  Interdependency  Team 

*  As  high  as  possible,  with  all  working  toward 
the  same  recognized  goal 

*  Goal-oriented  activities  involving  all  or 

subsets  of  members 

*  Shared  resources  (information,  KSAs) 

*  Sequenced  or  overlapping  procedures 

Situational  Factors 

*  Stable,  including  training 

*  Newly  formed  teams 

*  Training  is  seen  as  necessary  to  job 

performance  (high  motivation) 

*  Initial  skills  training  (prefer  team  training 
not  individual  skill  training) 

*  Able  to  observe  individual  and  team  learning 
and  improvement 

Performance  Measures 

*  Formative  as  well  as  summative  criteria 

available 

*  Quantitative  measures  available  or  can  be 

developed 

*  Able  to  track  process  and  to  identify 

intermediate  outcomes 

*  Able  to  distinguish  team  from  individual 

outcome  and  process  measures 
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APPENDIX  A  f continued! 


Secondary  characteristics 

Data  collection  Feasibility 

*  Team  composition 

*  Fleet  willingness  to  cooperate 

*  Task  structure 

*  Geographical  proximity 
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Date  ________________  Ship  Name _ 

Bay  _  Exercise  Name 

Session 


INSTRUCTIONS 

During  the  last  set  of  exercises  that  you  observed,  did  you  see  any  of  these  things 
happen  that  significantly  affected  work  outcomes?  If  so,  please 

(1)  X  the  positions  of  the  team  members  who  were  involved;  and 

(2)  Circle  the  X  of  the  indivldual(s)  who  did  what  you  marked. 

NOTE:  INS  -  Instructor;  EXT  ■  External  (e.g.,  Spotter,  Bridge,  etc.) 

In  the  first  column  marked  "Impact"  please  indicate  the  number  representing  the  level 
of  impact  that  each  of  these  incidents  had  on  the  team's  performance  using  the  scale  at 
the  top  of  each  page.  (1  *  No  Impact;  2  ■  Some  Impact;  and  3  ■  Major  Impact). 

In  the  last  column  on  the  sheets  for  Communication  and  Cooperation,  please  indicate 
the  frequency  with  which  the  team  member  performed  each  item  by  indicating  the  appropriate 
number  (1  m  Rarely;  2  ■  Sometimes;  3  ■  Regularly;  and  4  -  Consistently).  For  example,  if 
a  team  member  communicated  information  out  of  order  60Z  of  the  time,  you  would  put  a  three 

(3)  in  the  blank  beneath  the  column  marked  "Frequency."  , 

Important  events  which  happen  Infrequently  are  listed  on  the  page  entitled 
"Infrequent  Incidents."  Please  read  and  become  familiar  with  these  items.  It  is  only 
necessary  to  consider  this  page  when  a  listed  item  has  occurred. 

Finally,  the  last  page  is  reserved  for  any  additional  incidents  that  you  observe  that 
do  not  appear  elsewhere.  This  page  differs  from  the  previous  pages  because  it  is 
necessary  to  write  the  Incidents  in  the  blanks. 


TABLE  OF  CONTENTS 


Dimension  Page 

Communication  1 

Cooperation  2 

Team  Spirit  and  Morale  3 

Giving  Suggestions  or  Criticism  4 

Acceptance  of  Suggestions  or  Criticism  5 

Coordination  6 

Adaptability  7 

Infrequent  Incidents  8 

Additional  Incidents  9 
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COMMUNICATION 


1  ■  No  Impact 

2  -  Some  Impact 

3  ■  Major  Impact 


1  -  Rarely  (0  -252) 

2  *  Sometimes  (26-502) 

3  -  Regularly  (51-752) 

4  »  Consistently  (76-100* 


TAR 

R/T 

PLT 

TAL 

1.  Lowered  hl6  voice  and  mumbled 
when  communicating  Information 
to  other  team  members. 


2.  Communicated  information 
out  of  order. 


3.  Added  his  own  comments  to  the 
prescribed  commands,  thereby 
wasting  time. 


4.  Used  improper  terminology  when 
communicating  information. 


5.  Failed  to  ask  for  clarification 
on  a  communication  that  was 
unclear . 


6.  Asked  for  specific  clarification 
on  a  communication  that  was 
unclear. 


7.  Members  were  talking  among 
themselves  and  missed  a 
communication. 
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COOPERATION 


1  •  Rarely  (0  -252) 


1  ■  No  Impact  2  »  Sometimes  (26-502) 

2  •  Some  Impact  3  *  Regularly  (51-752) 

3  »  Major  Impact _ _ _ 4  ■  Consistently  (76-10 


n 

GLO 

CIC 

SUP 

NAV 

PLT 

TAR 

PLT 

R/T 

TAL 

R/T 

REC 

NAV 

REC 

INS 

EXT 

FREQUENCY 
a. 2,3, 4) 

1.  Checked  with  other  team 

members  when  uncertain  about 
what  to  do  next. 

B 

1 

1 

1 

I 

1 

B 

1 

1 

■ 

2.  Helped  another  member  who 
was  having  difficulty  with 
a  task. 

BB 

1 

1 

B 

1 

B 

B 

B 

B 

B 

a 

3.  Prompted  another  member  on 
what  he  had  to  do  next. 

B 

B 

l 

_  - 

■ 

s 

— 

B 

B 

B 

B 

a 

4.  Cave  suggestions  on  how  to 
do  a  task. 

B 

1 

1 

B 

1 

B 

8 

B 

B 

B 

a 

5.  Member  who  needed  assistance 
asked  for  help. 

B 

B 

S 

B 

1 

I 

B 

B 

B 

B 

a 

6.  Tried  to  push  another  member 
out  of  the  way  and  do  his 

Job  for  him. 

B 

B 

S 

B 

1 

B 

B 

E 

B 

B 

a 

gn 

7,  To  help  another  member, 
performed  a  task  that  was 
not  part  of  hla  job. 

B 

B 

1 

B 

s 

8 

B 

S 

B 

B 

a 

gjjjj 

8.  Was  uncertain  what  to  do  next 
and  failed  to  ask  for  help. 
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1  -  No  Impact 

2  -  Some  Impact 

3  ■  Major  Impact 


1.  Made  positive  statements  to 
motivate  the  team. 


2.  Patted  another  member  on  the 
back. 


3.  Assisted  another  member  when 
the  latter  had  a  difficult 
task  to  perform. 


4.  Discussed  ways  of  improving 
team  performance. 


5.  Formed  subgroups  or  cliques. 


6.  Argued  among  themselves. 


7.  Praised  another  member  for 
doing  well  on  a  task. 


8.  Made  negative  comments  or 
blamed  another  member  for 
the  failure  of  the  teas. 


9.  Made  a  joke  to  lighten  the 
tension. 


10.  Allowed  personality  conflicts 
to  interfere  with  work. 


11.  Made  negative  comments  about 
the  team  or  training. 


12.  Provided  moral  support  to 
a  member  who  had  made  a 
mistake. 


TEAM  SPIRIT  AND  MORALE 


IMPACT  A  CIC  NAV  TAR  R/T  R/T  NAV 

2.3)  GLO  GLO  SUP  PLT  PLT  TAL  REC  REC  INS  EXT 
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GIVING  SUGGESTIONS  OR  CRITICISM 


1  •  No  Impact 

2  •  Some  Impact 

3  •  Major  Impact 


1.  Raised  question  about 

Incorrect  procedure  used  by  a 
senior  member  of  the  team. 


2.  Called  attention  to  a  mistake 
made  by  another  member  • 
without  being  negative. 


3.  Noticed  a  mistake  and  did  not 
mention  it. 


A.  Asked  if  the  procedure  or 
information  was  correct  when 
he  wasn't  sure. 


S.  Suggested  to  another  member 
that  he  recheck  his  work  so 
that  he  could  find  his  own 
mistake. 


6.  Gave  unsolicited  and 

unnecessary  advice  to  another 
member. 


7.  Raised  his  voice  when 
correcting  another  member. 


8.  Verbally  reprimanded  another 
member  when  this  was  necessary, 


J.s'l] 

1',‘J 

■  1*1 

m 


IMPACT  A  CIC  NAV  TAR  R/T  R/T  NAV 

GLO  GLO  SUP  PLT  PLT  TAL  REC  REC  INS  EXT 
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ACCEPTANCE  0?  SUGGESTIONS  OK  CRITICISM 

l  •  No  Impact 


2  »  Some  Impact 


3  ■  Major  Impact 

IMPACT 

CLO 

A 

CLO 

CIC 

SUP 

NAV 

PLT 

TAR 

PLT 

R/T 

TAL 

R/T 

REC 

NAV 

REC 

INS 

EXT 

1.  Asked  vhat  he  had  done  wrong 
when  told  that  he  had  made  a 
mistake. 

s 

8 

8 

E 

8 

8 

8 

a 

8 

8 

a 

2.  Told  other  members  to  worry 
about  their  own  Jobs  and  let 
him  alone. 

s 

8 

8 

8 

8 

8 

8 

8 

8 

8 

i 

3.  Argued  with  another  member 
who  aald  he  had  made  a 
mistake. 

SB 

8 

8 

8 

8 

8 

8 

8 

8 

a 

a 

4.  Tried  to  cover  up  hie  own 
mlatake. 

SB 

8 

8 

8 

8 

8 

8 

8 

8 

8 

a 

5.  Thanked  another  member  for 
catching  his  mistake. 

me  am  am  m 

8 

8 

8 

8 

8 

8 

a 

8 

a 

a 

6.  Became  hostile  or  defensive 
when  criticized. 

■ 

8 

8 

8 

a 

a 

a 

i 

a 

a 

5 
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1.  When  finished  one  task, 
member  began  working  on 
another  task. 


2.  Coordinated  gathering  of 
required  information  in  an 
effective  manner. 


3.  Was  not  ready  with  information 
when  another  member  needed  ic. 


4.  Provided  information  that  was 
needed  before  being  asked  for 
it. 


5.  Was  ready  with  information 
when  other  members  needed  it. 


6.  Directed  members  on  what  to 
do  next. 


7.  Indicated  that  he  was  finished 
with  a  task  before  he  really 
was  so  that  he  could  beat  the 
clock. 


8.  When  not  busy  with  his  job, 
watched  what  the  other  members 
of  the  team  were  doing. 


9.  When  serving  as  a  backup  for 
another  member,  confirmed 
Information  without  checking  it. 


10.  Attempted  to  determine  the 
cause  of  discrepant 
information  before  going  on. 


11.  Failed  to  provide  information 
unless  asked. 


laohnioal  Report  »7-oi6 


1  •  Wo  lapse t 

2  •  lone  Xapact 

3  •  Hsjor  lapse t 


IMPACT  |  A  CXC  WAV  TAR  R/T  R/T  MAV 
IQLO  lUff  PLT  fLTlT 


l.  Honker  wss  unsbls  to  adapt  to 
information  provided  out  of 
order. 


2.  Performed  s  tssk  outside  of 
his  job  because  the  tesa 
needed  to  hsvs  the  work  dons. 


i,  Changed  the  wsy  ho  performed 
s  tssk  when  sekod  to  do  so. 


4.  Mode  no  otteapt  to  recover 
alsssd  Jnfoiastion. 


9.  Member  was  sbie  to  adept  to 
inforastion  provided  in  the 
wron|  order. 


4.  Mads  sure  he  had  all  of  tha 
inforastion  required  to 
eoaplsco  hie  job, 


7.  Provided  suggestion*  on  the 
best  wsy  to  U<iti  an  error, 


••  Refused  to  shenfo  the  wsy  he 
did  s  tssk  even  though  ho  woo 
doing  it  wrong, 
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INFREQUENT  INCIDENTS 

1  •  No  Impact 


2  "  Some  Impact 


3  «  Major  Impact 

IMPACT 

(1^2,3) 

GLO 

A 

GLO 

CIC 

SUP 

NAV 

PLT 

TAR 

PLT 

R/T 

TAL 

R/T 

REC 

NAV 

REC 

INsl 

EXT 

1.  Cave  a  different  Interpretation 
to  information  provided  by 
another  member  because  of  errori 
previously  made  by  that  member. 

9  3 

nHBH| 

1 

1 

I 

2.  Indicated  that  tie  knows  his 

Job  and  shouldn't  have  to  worry 
about  someone  else 'a  job. 

s 

e 

B 

B 

B 

g 

B 

B 

B 

— 

3.  Failed  to  assist  another 
member  who  was  having 
difficulty  and  let  him  fail. 

s 

s 

i 

8 

B 

B 

B 

B 

B 

B 

— 

4.  Member  became  overloaded  and 
failed  to  ask  for  assistance. 

B 

s 

B 

B 

B 

B 

B 

B 

B 

B 

B 

5.  Wrote  down  notes  for  another 
member  on  the  performance 
o i  the  letter's  job. 

— 

»  rnm 

i 

B 

B 

B 

B 

B 

8 

8 

B 

6.  While  waiting  for  Information 
from  another  member,  began 
tor  harass  the  other  member. 

BB 

s 

B 

B 

B 

B 

S 

B 

8 

B 

B 

7,  Ridiculed  a  p.flober  who  had 
made  a  mi  intake. 

— 

— 

mfSs 

B 

B 

B 

B 

fl 

B 

8 

!§ 

8.  Tried  to  cover  up  a  mistake 
made  by  another  member. 

1 

i 

1 

B 

a 

a 

a 

a 

a 

a 

8 
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1  *  Nr  Iapact 

2  -  Some  Iapact 

3  -  Major  Iapact 


g 


I 

* 

ft 


‘  *1  f=i  y»*  I* 
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OVERALL  TEAM 
PERFORMANCE 

INSTRUCTIONS:  Indicate  below  the  score  appropriate  for  rating  the  overall 
team  performance  up  to  this  point  in  the  training.  This  score  should 
reflect  current  team  performance  only.  Additionally,  please  provide  the 
information  concerning  strengths  and  weaknesses  in  the  place  provided. 


Team 

Team 

Team 

performance 

performance 

performance 

was  very 

was 

was  very 

poor 

average 

good 

5  4 

3 

2  1 

List  three  team  strengths: 

List  three  team  weaknesses: 

PLEASE  CIRCLE  SESSION:  PRE-MIDTERM  MIDTERM  POST-MIDTERM 

FINAL  EXAM  POST-TRAINING 
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APPENDIX  D 

Individual  Performance  Summary 
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INSTRUCTOR 

INDIVIDUAL  PERFORMANCE  SUMMARY 


INSTRUCTIONS: 

Upon  completion  of  the  current  training 

session. 

please  indica 

te  the  level 

of  each 

individual's  performance  for 

che  training  session  by  circling 

the  appropriate  score 

provided . 

This  performance  measure 

!  should 

reflect  the 

individual ' s 

performance  of 

his  assigned 

job  only  during  the  current 

training 

session. 

Member's 

Member *8 

Member’ s 

performance 

performance 

performance 

was  very 

was 

was  very 

poor 

average 

good 

TEAM  MEMBER 

GLO 

5 

4 

3 

2 

1 

AGLO 

5 

4 

3 

2 

1 

CIC  SUPER 

5 

4 

3 

2 

1 

NAV  PLOT 

5 

4 

3 

2 

1 

TARGET  PLOT 

5 

4 

3 

2 

1 

R/T  TALKER 

5 

4 

3 

2 

1 

R/T  RECORDER 

5 

4 

3 

2 

1 

BRIDGE  TALKER 

5 

4 

3 

2 

1 

PLEASE  CIRCLE  SESSION: 

PRE- 

■MIDTERM  MIDTERM 

POST-MIDTERM 

ITOftfjnn*VinHHfll'My.^JfW^WWreiCMiflt|rtfjn<Wtyy?y\?WWreyvy*y\/ErW^ni'*V WTVifVyu  WVfV>rwwiiWiTVr^  w^rurcwv»r\<Mvin4iTviTv»mniTur\iy\<<rvi 
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APPENDIX  E 

GLO  Individual  Performance  Summary 


nauwwt wmwnwwww  n  n  m  \m  aji  a*  n*  jvi  J%RJ^JUWlJWUWVUi^^wwvwv^  wvys/  vm  wjrvvm  irw  jryjrvw^Lrtv 


Technical  Report  87-016 

GLO 

INDIVJDMAL  PERFORMANCE  SUMMARY 

INSTRUCTIONS:  Upon  com  i.-t:  »n  of  the  current  training  session,  please 

indicate  the  level  of  t  ic!i  ,.  .dividual’s  performance  for  the  training 
session  by  circling  the  •  ipriate  score  provided.  This  performance 
measure  should  reflect  the  individual's  performance  of  his  assigned  job 
only  during  the  current  training  session. 


Member's 
performance 
was  very 
poor 


Member's 

performance 

was 

average 


Member '8 
performance 
was  very 
good 


TEAM  MEMBER 


AGLO 


CIC  SUPER 


NAV  PLOT 


TARGET  PLOT 


R/T  TALKER 


R/T  RECORDER 


BRIDGE  TALKER 


PLEASE  CIRCLE  SESSION:  PRE-MIDTERM  MIDTERM  POST-MIDTERM 


106 


Uf  iJin  wx^  wwm  w«f\»wvm vi  va  fvwi  .w  *v\  *v\  ns-  ka  k a  v\vi 


^.-■\  HjT  »t.n  *  *  HA  *  *  *  *  V.^V)l *  JTU X if j* 


Technical  Report  87-016 


APPENDIX  F 

Trainee  Self-Report  Questionnaire 


wiwwwirvwwu  Wi  wv  wv  «wm  ww wirw  wv-yvjty  t\*  jtv  it* 


,r*wvjrwry 


(Circle  On#) 


8IUET 
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NAV  Plot 


C1C  SUP 


R/T  Recorder 


NAV  Recorder 


Target  Plot 


R/T  Talker 


Thin*  about  the  exercises  in  the  last  training  session  in  which  you  too*  pert.  Circle  the  x  th*t  shows  how 
nuch  you  agree  or  disagree  with  each  statement.  If  you  ire  ’not  sure*,  circle  the  X  under  chat  category.  The' 
"team"  means  you  and  the  other  C1C  personnel  in  training  with  you  from  your  ship. 

The  only  people  who  are  going  to  see  your  answers  are  the  00U  researchers. 


A 

fc 

[IV 

tvl 

M 

E*  I 

Ivl 

E*M 


r?;<l 

» * .  I 


I* 


Strongly 

Disagree 


Disagree 


Strongly 

Agree 


Members  of  my  te-?m  knew  how  to  perform  their 
required  duties  in  this  set  of  exercises. 

Members  of  my  team  discussed  ideas  about  how 
to  proceed  in  this  set  of  exercises. 

Members  of  my  team  coopereted  with  each  other 
during  the  exercises. 

Members  of  my  team  did  not  do  their  best  in 
this  set  of  exercises. 

My  team  felt  that  the  success  of  our  group  was 
the  most  important  objective  of  this  exercise. 

Members  of  my  team  told  me  about  the  things  I 
needed  to  know  to  do  my  Job. 

Members  of  my  team  felt  under  pressure  during 
tni$  exercise. 

when  members  of  my  team  had  questions,  we 
could  turn  to  otners  for  neip. 

Members  of  my. team  had  confidence  In  th# 
accuracy  of  the  information  we  got  from  the 
Spotter,  bridge,  and  plot. 

Communications  were  not  always  clear  among 
members  of  my  team. 

The  activities  of  my  team  were  well  organlied. 

I  knew  exactly  what  I  was  supposed  to  do 
during  the  exercises. 


(OVER) 


-r*  -sp.  m  w 
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Strongly 

Oisigre* 

oi  store* 

Not 

Sure 

18Z3* 

u.  Th*  f  1n*l  outcoMl  of  this  set  Of  *k*rds*s 
were  mostly  the  rw*'t  of  rf»*t  our  teim 
«c«b«rs  did;  not  *Mt  other  people  did. 

X 

X 

X 

X 

1«.  Ny  teim  sent  iceurit*  informitlon  to  th* 

spotter,  bridge,  *nd  plot  it  th*  ippropriit* 
times. 

X 

X 

X 

X 

15.  Ny  teem  felt  tint  the  success  of  indlvlduil 
members  wis  th*  most  Important  objective  of 
this  exercise. 

X 

X 

X 

X 

16.  Success  in  my  job  depended  heivlly  on  th* 
ictlons  of  other  teim  members. 

X 

X 

X 

X 

17.  it  took  too  long  to  coordlnit*  Information 

In  my  teim. 

X 

X 

X 

X 

18.  1  completely  understood  how  my  position 

fits  in  with  the  work  of  other  members  of 

the  teim. 

X 

X 

X 

X 

19.  In  this  set  of  exercises,  the  leeder  of  my 
teim  showed  thit  he  Is  concerned  ibout  the 
welfir*  of  the  teim  members. 

X 

X 

X 

X 

20.  I  wts  not  sitlsfled  with  my  turn's 
performance  on  these  exercises. 

X 

X 

•x 

X 

21.  This  set  Of  trilnlng  exercises  his 
improved  th*  performinc*  of  our  teim. 

V 

X 

X 

X 

Plus*  inswer  th*  next  two  questions  using  the  following  choices.  Put  th*  ippropriit* 
provided. 

1.  A&LO  2.  Miv  Recorder  3.  C 1C  Sup  X.  Tirget  Riot  5.  Instructor 

6.  R/T  Tilker  7.  Niv  Riot  8.  R/T  Recorder  9.  510 

22.  Which  indlviduil  w*s  th*  -most  viluibl*  pliyer-  on  th*  teim  during  this  set  of  exercises? 

(Choose  only  on*) 


23.  whicn  individual  wis  th*  'Wut  viluibl*  pliyer"  on  the  teim  dur^np  this  set  o'  ***rcis*$? 
(Choose  only  on*) 
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H£SL 

x 


x 


X 


X 

X 

X 


X 


X 

X 


th*  blink 


vy 

■jQ 

tJG 
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Inetructionei  The  following  information  will  reoain  confidential  and  la  for 
rcaearch  only.  Each  team  member  should  fill  in  all  questions  carefully  and 

completely,  _ 

INDIVIDUAL  INFORMATION 

WHAT  18  YOUR  CURRENT  RANK _  RATE  _ 

WHAT  IS  YOUR  BILLET  ON  THE  NAVAL  GUNFIRE  SUPPORT  TEAM?  _ 

CLO _  NAV  PLOTTER  _  R/T  TALKER  _ 

AGLO  _  TARGET  PLOTTER  _  R/T  RECORDER  _ 

C1C  SUPERVISOR _ NAV  RECORDER  _  OTHER  _ 

HOW  LONG  HAVE  YOU  BEEN  IN  THE  NAVY?  (CIRCLE  YEARS  AND  MONTHS) 

YEARS  <1123456789  10  11-15  >15  MONTHS  <11  23  4  )  6  7  8  9  10  11 

HOW  LONG  HAVE  YOU  HELD  YOUR  CURRENT  RANK  OR  RATE?  (CIRCLE  YEARS  AND  MONTHS) 
YEARS  <1  1  2  3  4  5  >5  MONTHS  <1  1  2  3  4  5  6  7  8  9  10  11 

HOW  LONG  HAVE  YOU  BEEN  IN  YOUR  CURRENT  COMMAND?  (CIRCLE  YEARS  AND  MONTHS) 

YEARS  <1  1  2  3  4  5  >5  MONTHS  (112  3  4  5  i  7  8  9  10  11 

HOW  LONG  HAVE  YOU  BEEN  ASSIGNED  TO  THE  NGFS  TEAM  AT  YOUR  CURRENT  COMMAND? 
(CIRCLE  YEARS  AND  MONTHS) 

YEARS  <1  1  2  3  4  5  >5  MONTHS  <1  1  2  3  4  5  6  7  8  9  10  11 

HOW  LONG  HAVE  YOU  HELD  YOUR  CURRENT  BILLET  ON  THE  NGFS  TEAM  AT  YOUR  CURRENT 
COMMAND?  (CIRCLE  YEARS  AND  MONTHS) 

YEARS  <1  1  2  3  4  5  >5  MONTHS  <1  l  2  3  4  5  6  7  8  9  10  11 

CHECK  ANY  OTHER  OP  THE  FOLLOWING  BILLETS  THAT  YOU  HAVE  HELD  ON  THIS  TEAM. 

CLO  _  NAV  PLOTTER  _  R/T  TALKER _ 

AGLO  _  TARGET  PLOTTER  _ 

CIC  SUPERVISOR  _  NAV  RECORDER  _  R/T  RECORDER _ 

WHICH  OP  THE  FOLLOWING  SCHOOLS  HAVE  YOU  ATTENDED  IN  THE  LAST  3  YEARS? 

A  SCHOOL  _  LMET  _ 

C  SCHOOL _  RADAR  NAV  _ 

IP  YOU  HAVE  PREVIOUSLY  COMPLETED  THIS  TRAINING  WITH  YOUR  CURRENT  COMMAND, 
INDICATE  WHEN  AND  THE  BILLET  HELD.  YOU  MAY  CHECK  MORE  THAN  ONE. 

BILLET 

IN  THE  LAST  6  MONTHS  _  _ 

BETWEEN  12  *  6  MONTHS  AGO _  _ 

BETWEEN  1  A  2  YEARS  AGO _  _ 

BETWEEN  2  A  3  YEARS  AGO _  _ 

OVER  3  YEARS  AGO _  _ 


WHILE  YOU  HAVE  BEEN  IN  YOUR  CURRENT  BILLET  ON  THIS  TEAM  HOW  OFTEN  HAVE  YOU 
TRAINED  WITH  THIS  TEAM?  (CHECK) 

MORE  THAN  TWICE  A  WEEK  _  l  TO  2  TIMES  A  QUARTER 

l  TO  2  TIMES  A  WEEK  ___  NEVER  _ 

l  TO  2  TIMES  A  MONTH  _  OTHER  _ ~ 
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Scoring  Protocol  for  NGFS  Qualification  Exercises 
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APPENDIX  H 

SCORING  PROTOCOL  FOR  NGFS  QUALIFICATION  EXERCISES 

Prepared  by 

Randall  L.  Oeer  &  Ben  B.  Morgan,  Jr. 


i 

S 


During  NGFS  training,  the  midterm  and  final  exam  exercises 
are  scored  according  to  procedures  outlined  in  COMNAVSURFLANTINST 
3570 . 2c,  article  301.  The  overall  strategy  of  the  scoring 
procedure  is  to  test  the  readiness  of  teams  with  respect  to  all 
the  important  aspects  of  shore  bombardment  and  Naval  Gunfire 
Support.  The  specific  objective  of  this  procedure  is  to  "  reward 
safe,  accurate  and  timely  fire  support,  and  to  penalize  [teams] 
for  unsafe  practices,  poor  accuracy  and  tardy  response"  (p.  III- 
l) .  Because  team  members  are  responsible  for  maintaining  the 
equipment  employed  in  Gunfire  Support,  teams  are  evaluated  on 
both  personnel  and  equipment  performance;  that  is,  no  waivers  are 
given  for  equipment  malfunction.  In  addition,  the  scoring 
procedure  places  great  emphasis  on  the  safety  of  friendly  troops. 
This  is  accomplished  by  assessing  a  separate  "Endangering 
Friendly  Troops  Penalty"  whenever  a  salvo  impacts  too  closely  to 
friendly  troop  positions. 

The  exercises  are  scored  by  the  training  instructor  using 
standardized  scoring  sheets  designed  to  address  each  specific 
exercise.  In  general,  the  scoring  focuses  on  three  major  scoring 
factors:  Gunfire  Support  procedures,  Communication  procedures, 
and  the  teams'  ability  to  make  timely  responses  during  the 
training  program.  For  each  exercise,  the  scoring  sheets  specify 
the  the  missions  that  will  take  place  in  the  testing  session, 
number  of  "Maximum  Credit"  points  allotted  for  each  of  several 
specific  penalty  areas,  penalties  to  bs  assessed  during  the 
exercise,  and  information  to  assist  the  instructor  in  calculating 
the  team's  overall  performance  score.  Thus,  teams  receive  point 
credits  based  on  the  aocuraoy  of  their  performance,  the 
timeliness  of  their  responses,  and  their  ability  to  use  proper 
Gunfire  support  and  Communication  procedures.  They  are 
penalized  for  demonstrating  inappropriate  behaviors  and  for  not 
demonstrating  appropriate  behaviors.  That  is,  up  to  the  limit 
established  by  the  "Maximum  Credit"  allotment,  the  instructor 
deducts  points  for  inaccurate,  inappropriate,  or  untimely 
performances.  Article  301  of  the  training  manual  provides 
detailed  descriptions  of  the  various  credits  and  penalties 
associated  with  specific  actions  or  lack  of  actions  a  team  may 
demonstrate  during  the  test  sessions. 

With  respect  to  the  Gunfire  Support  scoring  factor, 
penalties  may  be  assessed  for  specific  behaviors  such  as  the 
following:  opening  fire  without  command,  reporting  erroneous 
'ready'  commands,  failing  to  respond  to  executive  orders  to  fire, 
inability  to  use  support  equipment,  violating  a  'check  fire' 
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order,  failure  to  achieve  a  correct  air  burst,  a  fuze-setting 
error,  providing  inadequate  illumination,  and  failure  to  deliver 
a  full  battery  of  salvos*  Penalties  related  to  Communication 
procedures  include)  mispronunciation  of  letters  or  numerals, 
failure  to  read  back  reports  originated  by  the  spotter,  omission 
in  read  back  of  transmissions,  communication  security  violations, 
misuse  of  'over*  and  ‘out',  and  failure  to  report  according  to 
doctrine.  Additional  penalties  may  be  assessed  in  some  instances 
when  excessive  time  is  required  in  the  execution  of  the  behavior. 

Finally,  it  should  be  note  that  teams  engaoe  in  the  test 
exercises  only  after  they  have  shown  proficiency  in  the  required 
pre-test  exsroises.  The  team  must  successfully  demonstrate 
proficiency  in  the  Basies  and  Pre-Midterm  exsroises  prior  to 
beinc  given  the  Midterm  test.  Similarly,  it  must  demonstrate 
proficiency  in  the  Post-midterm  exercises  and  have  successfully 
passed  the  Midterm  test  prior  to  taking  the  Final  exam. 
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APPENDIX  I 

FACTOR  ANALYSIS  OF  THE  SELF-REPORT 
QUESTIONNAIRE  DATA 


This  appendix  presents  further  information  regarding  the 
factor  analytic  results  of  the  self-report  questionnaire. 
Presented  here  is  information  regarding  reliability,  factor 
structure  and  factor  key  development. 

Factor  Analysis  Reliability 

Table  1-1  provides  the  alpha  coefficients  for  the  factor 
analyses  for  each  phase. 


Table  1-1 

Coefficients  Alpha  for  the 
Self  Report  Questionnaire  by  Phase 

Coefficient 


Ehase  Alpha 

Basic  .81 

Premidterm  .86 

Midterm  .86 

Postmidterm  .89 

Final  .87 


EasfeQE  interpretation 

The  procedures  for  interpreting  the  factors,  presented  in 
this  section,  were  rather  straightforward.  ’'Key"  items  included 
for  a  particular  factor  were  those  having  a  loading  of  .40  or 
greater  on  that  factor.  Once  the  factor  composition  was 
determined  (after  varimax  rotation,  and  using  the  .40  cut-off) 
only  those  factors  which  accounted  for  greater  than  4.5%  of  the 
variance  were  included  in  subsequent  analyses.  Below  that  level 
factors  were  unstable  and  uninterpretable. 

Table  1-2  presents  the  loadings  for  each  factor.  The 
following  code  is  used  to  identify  Tvhich  factors  derive  from  a 
particular  pairing: 

(1)  The  first  (capital)  letter  of  each  factor  code 
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represents  the  phase-pairing  to  which  it  belongs  (i.e., 
A  denotes  Basics  with  Preaidterm;  B  denotes  Premidterm 
with  Midterm;  C  denotes  Midterm  with  Postmidterra;  and, 
D  denotes  Postmidterm  with  Final) . 

(2)  The  number  in  the  factor  code  represents  the  number  of 
the  factor  derived  from  the  particular  pairing. 

(3)  The  second  (lowercase)  letter  singles  out  the  phase  (in 

the  pair)  to  which  the  factor  belongs  (i.e.,  b  is 
Basics;  r  is  Premidterm;  m  is  Midterm;  o  is 
Postmidterm;  and,  f  is  Final) .  As  an  example,  if  a  "b" 
appears,  it  refers  to  a  factor  where  the  items  loading 
upon  it  came  from  the  administration  of  the 

questionnaire  during  the  Basics  phase.  In  the  A2-b 
example,  Mb"  signifies  that  the  items  with  appreciable 
loadings  on  the  second  factor  derived  from  the 
Basics-Premidterm  pairing  came  form  the  administration 
of  the  questionnaire  during  the  Basics  phase. 


Table  1-2  contains  those  items  with  loadings  of  at  least  .40 
on  more  than  one  phase  factor  combination  in  the  analysis  of  a 
given  pair  of  phases  (A,  B,  C,  D) .  Table  1-2  also  shows  the 
proportion  of  variance  accounted  for  by  each  factor.  It  should 
be  noted  that  in  some  cases,  a  factor  contains  items  from  two 
phases  (i.e.,  Bl-r,  lm  and  B4-r,  4-m) .  The  loadings  are  listed 
separately  for  each  phase,  but  the  proportion  of  variance 
accounted  for  is  listed  for  the  overall  factor. 

To  help  the  reader  interpret  this  innovative  adaptation  of 
factor  analysis  and  the  rather  complicated  Table  1-2,  several 
examples  will  be  provided  here. 

Under  the  B  Phases  heading,  the  first  factor  is  1-r,  1-m. 
This  factor  contains  items  from  both  the  r  Phase  (Premidterm)  and 
the  m  phase  (Midterm) .  To  facilitate  interpretation  the  item 
loadings  are  listed  in  separate  columns,  according  to  the 
training  ph&^e  from  which  they  originated.  For  example,  item  4 
from  the  r  Phase  and  from  the  m  Phase  both  loaded  on  the  first 
factor  (denoted  by  the  1  in  Bl-r,  1-m) .  This  should  not  be 
mistaken  for  an  overlapping  factor;  the  items  from  the  two  phases 
combined  in  B  are  part  of  the  same  factor  (which  accounts  for 
28.5%  of  the  variance  in  the  factor  analysis  of  the  B  phase 
responses) . 

To  reiterate,  the  definition  of  overlapping  loadings 
requires  loadings  of  .40  by  an  item  on  more  than  one  of  the 
factors  appearing  in  the  factor  analysis  of  the  phase 
combinations.  Thus,  the  item  l  loadings  that  appear  in  factor 
Bl-m  and  in  B2-m  are  regarded  as  overlapping  (denoted  by  the 
underline)  because  the  loadings  for  the  item  are  greater  than  .40 
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Table  1-2 


Significant  Factor  Loadings  for  Items  in  each  Factor 


* 

Item  No. 

Paired  Phases 

I 

t 


A  Phases  B  Phases 

C  Phases 

D  Phases 

** 

1-r  2-b  3-r  1-r  1-m  2-m  3-r  4-r  4-m 

l-o  2-m  3-m 

l-o  2-f  3-o  3-f 

1. 

.73 

-iA§ 

.81 

.49 

.49 

.55 

2. 

.54 

.80 

.62 

.45 

.58  .54 

3. 

.48 

.67 

.45 

.56  .66 

.64 

.51 

.64 

4. 

.64 

.55 

.63 

.60 

.53 

.44 

5. 

.41 

.59 

.52 

.53 

.55 

.55 

6. 

7. 

.49 

8. 

.75 

.58  .45 

.50 

.56 

.68 

9. 

.65 

.41 

.50 

.50 

.58 

.43 

.60 

.58 

10. 

.64 

.71  .70 

.41 

.67 

.62 

.  f  4 

11. 

.69 

.67 

.56  .67 

.80 

.61 

.75 

.74 

12. 

.59 

.46 

.61 

.59 

.55 

.50 

13. 

14. 

_i52 

.43 

.48 

.42 

.55 

-i4S 

.53 

.61 

15. 

16. 

17. 

.52 

.65 

.57  ,67 

.53 

.71 

.65 

18. 

.71 

.41 

.44 

.52 

.58 

19. 

.67 

.52 

.41 

.65 

20. 

.44 

.61 

.59  .60 

.60 

.61 

.69 

.48 

21. 

.75 

.57 

.64 

.74 

.61 

.69 

Percentage  of  Variance 

22.0 

7.6 

5.5 

28.5  8.0 

5.8  4.6 

26.0 

8.8 

6.0 

25.8 

8.4 

6.1 

Kiev;  *  A  -  Basic  Phase  with  Premidterm  Phase 
B  -  Premidterm  Phase  with  Midterm  Phase 
C  -  Midterm  Phase  with  Postmidterm  Phase 
D  -  Poetmidterrn  Phase  with  Final  Phase 


**  b  -  Basic  Phase 
r  »  Premidterm  Phase 
m  «  Midterm  Phase 
o  »  Poetmidterrn  Phase 
f  »  Final  Phase 


1  ■  first  factor  of  the  pairing 

2  *  second  factor  of  the  pairing 

3  -  third  factor  of  the  pairing 

4  »  fourth  factor  of  the  pairing 
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on  more  than  one  phase  factor  combination.  Furthermore,  since 
the  difference  between  the  loadings  was  less  than  .20,  the  item 
was  not  included  in  the  final  list  of  key  items.  Item  14  in 
factors  C2-m  and  C3-m  is  an  example  of  where  there  is  overlap, 
but  the  difference  between  the  loadings  is  greater  than  .20. 
Therefore,  only  the  loading  of  the  item  not  found  in  the  final 
list  of  factor  items  is  underlined  (.40). 

Based  on  the  results  presented  in  Table  1-2 ,  there  appear  to 
exist  two  strong  and  persistent  factors  (teamwork-centered  and 
taskwork-centered) ,  plus  a  late  emerging  three  item  factor  (team 
jelling) ,  which  account  for  significant  amounts  of  the  variance. 
Caution  must  be  exercised  when  interpreting  a  three  item  factor 
because  its  reliability  may  be  questionable.  However,  this 
suggests  that  futi  a  revisions  of  the  questionnaire  should 
include  more  items  r  lated  to  that  team-jelling  factor  because  it 
may  uncover  an  important  set  of  influences.  The  following 
discussion  considers  the  factors  that  derive  from  the  pairing  of 
phases. 

Basics  and  Premidterm  (A) .  In  the  A  phases,  there  are  three 
factors  which  satisfy  the  standard  established  for  proportion  of 
total  variance  accounted  for.  The  three  factors  accounted  for  a 
total  of  35.1%  of  the  variance  with  a  single  Phase  b  factor  (2-b) 
and  two  Phase  r  factors  (1-r  and  3-r) . 

Factor  2-b  (the  basics-skill  factor)  accounted  for  7.6%  of 
the  variance.  Among  the  items  included  (1,  2,  3,  5,  9,  11,  14, 
17,  and  20)  are  those  relating  to  aspects  of  performance, 
coordination,  cohesion,  and  communication.  In  the  Basics  phase, 
the  first  stage  of  training,  it  appears  that  members  perceive  the 
team  to  be  confronting  a  broad  spectrum  of  learing  requirements 
including  how  to  work  together  as  a  team,  as  well  as  how  to 
perform  their  basic  technical  task  functions.  This  basics -skill 
factor  reflects  both  team  forming  activities  (items  2,  3,  9,  11, 
14,  and  17;  coordination,  cohesion,  communication)  and  task 
performance  aspects  (items  1,  5,  and  20) .  The  relevance  of  these 
aspects  at  this  stage  is  supported  by  the  demographics  data  and 
instructor  reports  indicating  that  often  half  or  more  of  the 
members  are  new  to  the  team  when  the  training  begins.  The 
factors  emerging  in  the  Basics  phase  seem  to  reflect  dynamic 
processes  that  operate  to  bring  the  team  members  to  a  common 
level  of  skill  so  that  they  can  begin  training  for  specific 
gunfire  support  missions. 

There  were  two  Phase  r  factors  (1-r  and  3-r)  emerging  from 
this  analysis.  One  factor  (1-r)  can  be  called  a  '’teamwork- 
centered"  factor.  Factor  1-r  consists  of  items  (2,  3,  8,  9,  12, 
and  18)  that  deal  with  coordination,  communication,  and  cohesion. 
Essentially,  these  items  deal  with  those  perceptions  of  the  team 
members  bearing  upon  activities  that  involve  working  with  or  for 
other  team  members  and  that  demonstrate  interdependence. 
Representative  of  this  factor  are  items  (e.g.,  2  and  8) 
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reflecting  the  degree  to  which  one's  work  behaviors  fit  in  with 
the  activities  of  other  members;  like  turning  to  other  team 
members  for  help,  and  discussing  ideas  about  how  to  proceed  in 
the  firing  mission  exercises.  This  factor  accounted  for  22.0%  of 
the  variance. 

The  second  factor  (3-r)  can  be  labelled  a  "taskwork- 
centered"  factor.  It  accounted  for  5.5%  of  the  variance.  As 
contrasted  with  the  teamwork  factor,  the  taskwork  factor  is  made 
up  of  items  that  deal  with  organization  and  performance  of  work 
tasks.  For  example,  items  11,  17,  and  20  reflect  the  degree  to 
which  the  activities  of  the  team  were  well  organized,  the  amount 
of  time  it  took  to  coordinate  information,  and  the  evaluation  of 
team  member  performance.  It  can  be  seen  that  the  items  which  are 
part  of  this  factor  pertain  to  the  process  of  organizing  to 
complete  the  task  as  well  as  achieving  the  desired  performance 
outcomes . 

Of  particular  interest  is  the  fact  that  the  teamwork  factor 
accounts  for  the  greatest  amount  of  variance  of  the  A  factors. 
This  provides  empirical  f.upport  for  the  assumption  that 
development  of  "teamwork"  and  "teamness"  are  perceived  as 
essential  elements  of  a  properly  functioning  team.  It  appears 
noteworthy  that  early  in  the  evolution  of  the  team,  an  emphasis 
is  placed  on  the  development  of  "teamness".  This  may  in  part 
represent  the  "newness"  of  many  of  the  team  members  inasmuch  as 
they  must  learn  how  to  work  with  each  other  in  order  to  carry  out 
particular  tasks .  The  amount  of  practice  the  team  has  had  before 
arriving  at  NGFS  also  may  affect  the  importance  of  the  "teamness" 
factor  in  this  early  pairing.  For  example,  if  a  team  did  not 
practice  NGFS  before  arrival  at  the  simulator,  relatively  more 
time  would  need  to  be  spent  developing  team  skills,  compared  to  a 
team  arriving  at  -he  trainer  well  practiced.  In  the  latter  case, 
the  "teamness"  m  ght  already  exist.  Reports  from  instructors 
indicate  that  th  >.  norm  is  for  a  team  to  have  little  or  no 
practice  working  together  before  arrival  at  the  school. 

Premidterm  and  Midterm  (B)  .  Four  factors,  including  two 

which  contain  items  from  more  than  a  single  phase  (1-r.  1-m,  2-m, 
3-r,  and  4-r,  4-m)  resulted  from  this  analysis  of  B  phases.  The 
first  factor  (1-r,  1-m),  a  taskwork  factor,  contains  items  from 
both  Phases  r  and  m  and  accounts  for  28.5%  of  the  variance.  A 
clearer  explanation  of  this  factor  is  achieved  by  separating  the 
terns  from  the  two  phases,  1-r  and  1-m.  For  1-r  the  items  in 
this  taskwork  factor  (items  4,  10,  11,  17,  and  20)  are  identical 
to  those  found  in  A3-r  from  the  previous  pairing.  The  items  from 
1-m  in  the  taskwork  factor  (items  3,  10,  11,  14,  17,  and  20)  are 
similar  to  those  found  in  1-r.  Where  differences  occur, 
specifically  the  addition  of  two  additional  items5  (3  and  14) ,  the 
loadings  on  those  items  are  the  lowest  for  the  factor.  These 
additional  items  may  indicate  the  beginning  emergence  of  a 
maturation  or  integration  process  in  the  team  that  facilitates 
successful  completion  of  tasks. 
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The  second  and  third  factors  (2-m  and  3-r)  both  reflect 
aspects  of  teamwork-centered  activities  accounting  for  8.0%  and 
5.8%  of  the  variance  respectively.  These  two  factors  are  quite 
similar.  One  new  item  (5)  in  3-r,  concerned  with  whether  or 
not  the  team  felt  that  the  success  of  the  group  was  the  most 
important  objective  of  the  exercise,  may  reflect  the  differences 
between  the  two  phases.  Phase  r  can  be  regarded  as  a  learning 
phase  in  which  team  members  are  learning  to  carry  out  specific 
missions.  Phase  m,  on  the  other  hand,  is  definitely  a 
performance  evaluation  phase.  The  team  must  successfully 
complete  the  Midterm  phase  in  order  to  move  ahead  to  complete  the 
training  course.  It  is  not  surprising  that  success  of  the  group 
is  perceived  by  its  members  as  crucial  here,  given  the  testing 
situation. 

Factor  4-r,  4-m  contains  items  from  both  Phase  r 
(Premidterm)  and  Phase  m  (Midterm) .  This  factor  accounts  for 
4.6%  of  the  variance  and  contains  items  that  show  the  members  to 
be  acknowledging  increasingly  close  collaboration,  affiliation, 
cohesion,  and  mutual  supportiveness.  Items  2,  3,  and  8  reflect 
exchange  of  ideas,  turning  to  others  for  help,  and  other  forms  of 
cooperation.  These  reflect  what  the  instructors  refer  to  as 
"jelling",  a  process  that  may  be  accelerated  here  because  the 
Midterm  phase  (Phase  m)  is  a  cruical  point  in  the  life  cycle  of 
the  team. 

Perhaps  the  most  significant  difference  between  the  A  Phases 
(b  and  r)  analysis  and  th  B  Phases  (r  and  m)  analysis  is  that  in 
the  former  the  teamwork  factor  accounted  for  most  of  the 
variance,  while  in  the  latter,  taskwork  variables  accounted  for 
most  of  the  variance.  This  may  be  due  in  part,  to  the  team 
approaching  a  testing  situation.  Additionally,  this  findings 
suggest  that  a  certain  level  of  development  of  teamwork  skills 
must  be  attained  before  the  full  potential  for  development  of  the 
team's  taskwork  skills  can  be  realized. 

Midterm  and  Postmldterm  (C) .  This  C  Phases  iteration 
produced  three  factors  (l-o,  2-m,  and  3-m) .  Factors  2-m  (items 
3,  4,  10,  11,  14,  17,  and  20)  and  3-m  (items  5,  9,  12,  18,  19, 
and  21)  again  represent  taskwork  and  teamwork,  accounting  for 
8.8%  and  6.0%  of  the  variance,  respectively.  These  factors  are 
identical  in  item  makeup  with  those  found  in  the  previous  pairing 
(B  Phases)  for  the  factors  containing  items  resulting  from  the 
Midterm  phase.  That  is,  factors  Bl-m  and  B2-m  contain  the  same 
items  as  C2-m  and  C3-u,  though  there  are  some  minor  differences 
in  the  loadings. 

Factors  l-o  accounts  for  the  largest  percentage  of  the 
variance,  26.0%.  This  factor  is  different  than  previous  factors. 
There  is  a  convergent  transformation;  there  is  no  longer  a  clear 
separation  of  teamwork  and  taskwork  items  (l,  2,  3,  4,  5,  8,  9, 
10,  11,  12,  14,  17,  18,  19,  20,  and  21).  A  merging  of  the  two 
factors  into  a  single  factor  has  occured.  The  items  that  form 
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the  taanwork  factor  (3 -a)  and  taskwork  factor  (2 -a)  ara  all  found 
in  factor  l-o;  tha  merged  factor.  Soao  of  tha  itaaa  (2,  8)  in  1- 
o  not  found  in  2-a  and  3-a  can  ba  found  in  B4-a  tha  taaa  jailing 
factor. 

Poatnidtam  and  Pinal  IQI*  Xn  tha  D  Phaaaa  analysis/  thraa 
factors  aaarga  that  account  for  more  than  S%  of  tha  varianes/ 
including  a  factor  containing  items  from  two  phases  (l-o,  2-f,  3- 
o,  3-f ) .  Two  of  tha  footers  l-o  (itans  1/  4,  8,  7,  9,  10,  11, 
12/  14,  17,  20,  and  21)  and  2-f  (itans  1,  3.  4,  8,  8,  9,  10,  11, 
14,  20,  and  21)  ara  tha  nargad  factors  sinilar  to  Cl-o  found  in 
tha  previous  iteration. 

Factor  3-o,  3-f,  accounting  for  6.1%  of  tha  variance,  is 
nada  up  of  two  items,  one  fron  Phase  o  (item  8)  and  one  fron 
Phase  f  (itan  2) .  Tha  itans  ara  vary  similar  to  those  which 
comprise  tha  jailing  factor  (B4-r,  4-n)  in  tha  B  Phases  analysis. 
Tha  similarity  between  Phases  r  and  n  and  Phases  o  and  f  is  that 
both  of  than  include  tasting  situations.  Xt  nay  ba  that  whan 
taans  ara  being  tasted,  tha  need  for  tha  team  to  jail— to  bacons 
nora  cohesive  and  mutually  supporting— becomes  nora  compelling, 
so  that  loadings  concentrate  in  separata  factors  rather  than 
being  assinilatad  as  part  of  tha  taanwork  factor  as  in  tha  A 
Phases. 

A  summary  of  tha  proportion  of  variance  accounted  for  by 
each  factor  in  tha  respective  phases  is  provided  by  Table  1-3. 


Table  X-3 

Bunnary  Of  Varianoe  Accounted  For  By  Factors 
Identified  For  Each  Phase  Of  Team  Training 


Factor  identification 

TRAINING  PHASE  TEAMWORK  TASKWORK  TIAM/TASX  JELLING 


Basics 

Fre-Midtera 

22.0 

9.5 

7.6 

8.8 

Pre-Midterm/Midterm 

Midterm 

8.0 

8.8 

4.6 

Post-Midtarn 

26.0 

25.8 

Post -Midterm/Final 
Final 

8.4 

6.1 
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cflMtmiating  ttfltilii  ol  gargiPtloni 

Tabla  1-4  presents  tha  data  in  Tabla  1-2  in  a  form  designed 
to  make  oaaior  tha  comparison  of  faotora.  It  raprasants  a  atap 
in  tha  direction  of  constructing  a  anall  number  of  keys  that 
could  ba  uaad  to  provide  factor  profile  aooraa  for  individuale. 
Thaaa  profiles  could  than  ba  ar~'egated  to  provide  profile!  of 
teams  that  show  atatua  and/or  change  at  different  pointa  in  time 
(i.s.,  phaaaa  of  training)  baaed  upon  tha  Trainee  salf-Raport 
Questionnaire  responses  which  the  instruct ional  staff  oould  use 
in  tailoring  training  to  fit  the  needs  of  teams  at  different 
points  in  their  development.  There  follows  a  description  of  how 
suoh  keys  were  built. 

To  begin  with/  guidelines  were  established  to  maximize  the 
independence  of  the  factors  (i.a.,  minimize/  insofar  as  possible, 
the  correlations  among  the  dimensions  measured  by  the  factor 
keys).  A  first  sort  identified  items  that  had  loadings  of  .40  or 
above.  Where  there  was  "overlap,"  loadings  of  .40  and  above  on 
two  factors  in  the  same  phase  or  a  given  analysis  (underlined  in 
Table  1-2)/  the  rule  Imposed  was  that  when  the  dlfferenoe  in 
loadings  on  the  two  faotora  was  .20  or  more  the  item  would  be 
assigned  to  the  key  of  the  factor  on  which  it  had  the  higher 
loading.  Items  that  did  not  meet  these  requirements  were  dropped 
from  tna  pool,  Those  that  remained  are  shown  in  Table  X-4. 

Inspection  of  Table  1-4  led  to  the  establishment  of  three 
keys  based  upon  distinguishable  patterns  of  items.  The  teamwork- 
centered  profile  consists  of  items  8,  9,  12,  is,  19,  and  21. 
(See  Appendix  F  for  a  copy  of  the  Trainee  Self-Report 
Questionnaire  items) .  These  items  were  chosen  for  the  profile 
beoause  they  were  generally  found  in  the  teamwork-oentered  factor 
aoroas  ail  pairings.  The  profile  for  the  task-centered  factor 
consists  of  items  4,  10,  11,  17,  and  20.  The  profile  for  the 
final  factor  only  appears  at  the  Premidterm  and  Midterm  pairing. 
Yet,  the  amount  of  variance  that  it  accounts  for  indicates  that 
it  is  relevant  to  the  change  in  the  teams  over  time.  The  items 
oontained  in  this  profile  are  2,  3,  and  6.  This  is  the  team 
"jelling**  factor.  Oiven  that  these  are  only  three  items  in  this 
factor,  similar  items  oould  be  added  to  future  revisions  of  the 
Trainee  Self-Report  Questionnaire  in  order  to  increase 
reliability. 


125 


Technical  Report  87-016 


Table  1-4 

Items  by  Item  Number  in  each  Factor 
After  Deletion  of  Overlaps 


* 

Faired  Biases 


A  Phases 

3  Phases 

C  Phases 

D  Phases 

** 

l-r  2-b  3-r 

l-r  1-m  2~m  3-r  4-r  4-m 

l-o  2-m  3-m 

l-o  2— f  3-o  3-f 

1 

1 

1 

1 

2 

2 

2 

2 

2 

3 

3 

3 

3 

3 

3 

3 

3 

4 

4 

4 

4 

4 

4 

5 

5 

5 

5 

5 

5 

7 

8 

8 

8 

8 

8  8 

9 

9 

9 

9 

9 

9 

9 

9 

10 

10 

10 

10 

10 

10 

10 

11 

11 

11 

11 

11 

11 

11 

11 

12 

12 

12 

12 

12 

12 

14 

14 

14 

14 

14 

14 

14 

17 

17 

17 

17 

17 

17 

17 

18 

18 

18 

18 

18 

19 

19 

19 

19 

20 

20 

20 

20 

20 

20 

20 

7^ 

21 

21 

21 

21 

21 

r 

d*  <4. 

Keyi  *  A  *  Basic  Blase  with  Premidtann  Biase 
B  -  Premidtann  Biase  with  Midterm  Phase 
C  -  Midterm  Phase  with  Postmidterm  Phase 
D  -  Postmidterm  Phase  with  Fined  Phase 


**  b  -  Basic  Phase 
r  -  Premidterm  Phase 
m  -  Midterm  Phase 
o  -  Postmidterm  Phase 
r  ■  Final  Phase 


1  -  first  factor  of  the  pairing 

2  -  second  factor  of  the  pairing 

3  -  third  factor  of  the  pairing 

4  -  fourth  factor  of  the  pairing 
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